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Abstract — We consider the distributed source coding system 
of L correlated Gaussian sources Yi,l = 1,2, ■•■,L wiiicii 
are noisy observations of correlated Gaussian remote sources 
Xfc, fc = 1, 2, ■ ■ ■ , A'. We assume that Y^ = \Yi,Y2, ■ ■ ■ ,Yl) is 
an observation of the source vector X'"^ — '(Xi,X2, • ■ • ,Xk), 
having the form Y^ — AX'^ +N'^, where A is a L x A' matrix and 
N^ — ^{Ni, N-z,- ■ ■ , Nl) is a vector of L independent Gaussian 
random variables also independent of X'^ . In this system L 
correlated Gaussian observations are separately compressed by 
L encoders and sent to the information processing center. We 
study the remote source coding problem where the decoder at 
the center attempts to reconstruct the remote source X'^ . We 
consider three distortion criteria based on the covariance matrix 
of the estimation error on X'^ . For each of those three criteria 
we derive explicit inner and outer bounds of the rate distortion 
region. Next, in the case of A = L and ^ = Jl, we study the 
multiterminal source coding problem where the decoder wishes 
to reconstruct the observation Y'" — X^ + N^. To investigate 
this problem we shall establish a result which provides a strong 
connection between the remote source coding problem and the 
multiterminal source coding problem. Using this result, we drive 
several new partial solutions to the multiterminal source coding 
problem. 

Index Terms — Multiterminal source coding, rate distortion 
region, CEO problem. 



I. Introduction 

Distributed source coding systems of correlated informa- 
tion sources are a form of communication system which is 
significant from both theoretical and practical points of view 
in multi-user source networks. The first fundamental theory 
in those coding systems was established by Slepian and Wolf 
|[T]. They considered a distributed source coding system of 
two correlated information sources. Those two sources are 
separately encoded and sent to a single destination, where 
the decoder wishes to decode the original sources. In the 
above distributed source coding systems we can consider a 
situation where the source outputs should be reconstructed 
with average distortions smaller than prescribed levels. This 
situation yields a kind of multiterminal rate distortion theory in 
the framework of distributed source coding. The rate distortion 
region is defined by the set of all rate vectors for which 
the source outputs are reconstructed with average distortions 
smaller than prescribed levels. The determination problem of 
the rate distortion region is often called the multiterminal 
source coding problem. 

The multiterminal source coding problem was intensively 
studied by E1-|II2|. Wagner and Anantharam llQi gave a new 
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method to evaluate an outer bound of the rate distortion region. 
Wagner et al. |[TTI| gave a complete solution to this problem 
in the case of Gaussian information sources and quadratic 
distortion by proving that the sum rate part of the inner bound 
of Berger lU and Tung is optimal. Wang et al. [|l2l gave a 
new alternative proof of the sum rate part optimality. In spite of 
a recent progress made by those three works, the multiterminal 
source coding problem still largely remains open. 

As a practical situation of the distributed source coding 
system, we can consider a case where the distributed encoders 
can not directly access the source outputs but can access 
their noisy observations. This situation was first studied by 
Yamamoto and Ito ifTSl . They call the investigated coding 
system the communication system with a remote source. 
Subsequently, a similar distributed source coding system was 
studied by Flynn and Gray lfT4ll . 

In this paper we consider the distributed source coding 
system of L correlated Gaussian sources Yi,l ~ 1, 2, • • • , i 
which are noisy observations of Xfe,fc = 1,2,- ■ ■ ,K. We 
assume that Y^ = *(Yi,Y2, •••,Yl) is an observation of 
the source vector X^ = *(Xi,X2, • • • , Y^ ), having the 
form Y^ = AX'^ + N^ , where A is a L x K matrix and 
N^ = '(Yi, Y2, • • • , Nl) is a vector of L independent Gaus- 
sian random variables also independent of Y^^. In this system 
L correlated Gaussian observations are separately compressed 
by L encoders and sent to the information processing center 
We study the remote source coding problem where the decoder 
at the center attempts to reconstruct the remote source X^ . 

We consider three distortion criteria based on the covari- 
ance matrix of the average estimation error on X^. The 
first criterion is called the distortion matrix criterion, where 
the estimation error must not exceed an arbitrary prescribed 
covariance matrix in the meaning of positive semi definite. 
The second criterion is called the vector distortion criterion, 
where for a fixed positive vector D^ = {Di,D2, ■ ■ ■ ,Dk) 
and for each k ~ 1,2,- ■ ■ ,K, the diagonal (fc, fc) element 
of the covariance matrix is upper bounded by Dk- The third 
criterion is called the sum distortion criterion, where the trace 
of the covariance matrix must not exceed a prescribed positive 
level D. For each distortion criterion the rate distortion region 
is defined by a set of all rates vectors for which the estimation 
error does not exceed an arbitrary prescribed distortion level. 

For the first distortion criterion, i.e., the distortion matrix 
criterion we derive explicit inner and outer bounds of the 
rate distortion region. Those two bounds have a form of 
positive semi definite programming with respect to covariance 
matrices. Using this results, for each of the second and third 
distortion criteria we derive explicit inner and outer bounds 
of the rate distortion region. In the case of vector distortion 



criterion our outer bound includes that of Oohama Il22l as a 
special case by letting K — L and A = IlAvi the case of sum 
distortion criterion we derive more explicit outer bound of the 
rate distortion region having a form of water filUng solution. 
In this case we further show that if the prescribed distortion 
level D does not exceed a certain threshold, the inner and 
outer bounds match and derive two different thresholds. The 
first threshold improves the threshold obtained by Oohama 
Il23l . ll24ll in the case of K = L,A ~ I^. The second threshold 
improves the first one for some cases but neither subsumes the 
other. 

When K = 1, the distributed source coding system treated 
in this paper becomes the quadratic Gaussian CEO problem 
investigated by lfT2l . ifTSI - lfTSl . The system in the case of K = 
L and sum distortion criterion was studied by Pandya et al. 
|fT9l . They derived lower and upper bounds of the minimum 
sum rate in the rate distortion region. Several partial solutions 
in the case of K — L, A — II, and sum distortion criterion 
were obtained by ll20l - ll24l . The case of K = L, A ~ II, and 
vector distortion criterion was studied by ||22l . 

Recently, Yang and Xiong [261 have studied the same 
problem. They have derived two outer bounds of the rate 
distortion region in the case of sum rate distortion criterion. 
When K — L,A ~ II, the first outer bound does not coincide 
with the outer bound obtained by Oohama ||2T1 - Il24l . When 
*'AA ~ Ik, they have obtained the second outer bound tighter 
than the first one. This bound is the same as that of our 
result of this paper. When *j4yl — Ik, Yang et al. Il27l have 
derived a threshold on the distortion level D such that for D 
below this threshold their second outer bound is tight. Their 
threshold also improves that of Oohama 12311 . Il24l in the case of 
K = L,A ^ II. Comparing the formula of our first threshold 
with that of and Yang et al. Il27ll . we can see that we have no 
obvious superiority of either to the other On the other hand, 
our second threshold is better than their threshold for some 
nontrivial cases. 

In this paper, in the case of K ~ L and A = II, we study 
the multiterminal source coding problem where the decoder 
wishes to reconstruct the observation Y^ ~ X^ + N^ . Simi- 
larly to the case of remote source coding problem, we consider 
three types of distortion criteria based on the covariance matrix 
of the estimation error on Y^. Based on the above three 
criteria, three rate distortion regions are defined. 

The remote source coding problem is often referred to as 
the indirect distributed source coding problem. On the other 
hand, the multiterminal source coding problem in the frame 
work of distributed source coding is often called the direct 
distributed source coding problem. As shown in the paper 
of Wagner et al. lITTl and in the recent work by Wang et 
al. fV2\ . we have a strong connection between the direct and 
indirect distributed source coding problems. To investigate the 
determination problem of the three rate distortion regions for 
the multiterminal source coding problem we shall establish a 
result which provides a strong connection between the remote 
source coding problem and the multiterminal source coding 
problem. This result states that all results on the rate distortion 
region of the remote source coding problem can be converted 
into those on the rate distortion region of the multiterminal 



source coding problem. Using this relation and our results on 
the remote source coding problem, we drive new three outer 
bounds of the rate distortion regions for each of three distortion 
criteria. 

In the case of vector distortion criterion, we can obtain a 
lower bound of the sum rate part of the rate distortion region 
by using the established outer bound in this case. This bound 
has a form of positive semidefinite programming. By some 
analytical computation we can show that this lower bound is 
equal to the lower bound obtained by Wang et al. IIT2I and 
tight when L = 2. Our method to derive this result essentially 
differs from the method of Wang et al. [12j. It is also quite 
different from that of Wagner et al. ifTTl . Hence in the case 
of two terminal Gaussian sources there exists three different 
proofs of the optimality of the sum rate part of the inner bound 
of Berger g) and Tung ||S|. 

In the case of sum distortion criterion we derive an explicit 
threshold such that for the distortion level D below this 
threshold the outer bound coincides with the inner bound. An 
important feature of the multiterminal rate distortion problem 
is that the rate distortion region remains the same for any 
choice of covariance matrix S^i and diagonal covariance 
matrix Yjj^l satisfying Eyi, = Yix^+^f^L. Using this feature, 
we find a pair (Ex^j ^n^) which maximizes the threshold 
subject to YjyL = E^i, + S^i,. 

Let t{Y^) = {Y2,Y3,---,YL,Yi)he a cyclic shift of the 
source F^ = {Yi,Y2,Y3, ■ ■ ■ ,Yl). We say that the source 
Y^ has the cyclic shift invariant property if the covariance 
matrix I]^(yi,) of t{Y^) is the same as the covariance matrix 
YiyL of Y^. When Y^ has the cyclic shift invariant property, 
we investigate the sum rate part of the rate distortion region. 
We derive an explicit upper bound of the sum rate part from 
the inner bounds of the rate distortion region. On a lower 
bound of the sum rate part we derive a new explicit bound by 
making full use of the cyclic shift invariance property of Eyt . 
We further derive an explicit sufficient condition for the lower 
bound to coincide with the upper bound. We show that the 
lower and upper bounds match if the distortion does not exceed 
a threshold which is a function of Syt and find an explicit 
form of this threshold. As a corollary of this result, in the case 
of vector distortion criterion we obtain the optimal sum rate 
when Y^ is cyclic shift invariant and Z?^ has L components 
with an identical value D below a certain threshold depending 
only on Syi 

II. Problem Statement and Previous Results 

A. Formal Statement of Problem 

In this subsection we present a formal statement of problem. 
Throughout this paper all logarithms are taken to the base 

natural. Let Aa' = {1, 2, • • • , i^} and Al = {1, 2, • • • , L}. 
Let Xk , k G Ak be correlated zero mean Gaussian random 
variables. For each k g A^-, Xk takes values in the real 
line M. We write a K dimensional random vector as X^ ~ 
'(ATi, X2, • • • , Xk)- We denote the covariance matrix of A"^ 

by ^xK- Let Y'^ = '{Fi, F2, • • • , 1l) be an observation of 
the source vector X^, having the form Y^ = AX^ + N^, 

where ^ is a L x AT matrix and N^ = '(A^i, A^2, • • ■ , Nl) 
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Fig. 1. Distributed source coding system for L con'elated Gaussian 

observations 



is a vector of L independent zero mean Gaussian random 
variables also independent of X^ . For I e A^, aj^ stands for 
the variance of Ni. Let {(Xi(t), X2{t), ■ ■ ■,XK{t))}?^i be 
a stationary memoryless multiple Gaussian source. For each 
t = 1,2,- ••, X^{t) ^\Xi{t),X2{t),---,Xk{t)) has the 
same distribution as X^ . A random vector consisting of n 
independent copies of the random variable Xk is denoted by 

Xfc = (Xfe(l),Xfe(2),---,Xfe(n)). 

For each t = 1,2,---, Y^{t) ^\Yi{t), FaW, • • • ,>l W) is 
a vector of L correlated observations of X^it), having the 
form Y^{t) = AX^{t)+N^{t), where N^{t),t = 1,2,---, 
are independent identically distributed (i.i.d.) Gaussian random 
vector having the same distribution as N^. We have no 
assumption on the number of observations L, which may be 
L>K 01 L< K. 

The distributed source coding system for L correlated 
Gaussian observations treated in this paper is shown in Fig. 
[T] In this coding system the distributed encoder functions 
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where | |a| | stands for the Euclid norm of n dimensional vector 
a and (a, b) stands for the inner product between a and b. 
Let E ^-jvA' be a covariance matrix with dkk' in its (fc, fc') 



element. Let E^ be a given K x K covariance matrix which 
serves as a distortion criterion. We call this matrix a distortion 
matrix. 

For a given distortion matrix Yid, the rate vector {Ri, 
R2r ' ' :Rl) is 'Sid-admissible if there exists a sequence 
{(^("\ ^^"V • • , V^l\ ^^"^)}^=i such that 

limsupi?!"^ <Ru forleAL, 

n— ^-OD 

limsup il]^^,_^K ^ Ed , 

n— >-oo 

where ^1 ^ A2 means that A2 — Ai is a positive semi- 
definite matrix. Let TZLi^dl'^xKY^) denote the set of all E^- 
admissible rate vectors. We often have a particular interest in 
the minimum sum rate part of the rate distortion region. To 
examine this quantity, we set 



^sum,L(Srf|ExKYL) 
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{Ri,R2.,---,Rl) 




We consider two types of distortion criterion. For each distor- 
tion criterion we define the determination problem of the rate 
distortion region. 

Problem 1. Vector Distortion Criterion: Fix K x K 
invertible matrix T and positive vector D^^ = {Di, Z?2, • • • 
, Dk)- For given F and D^ , the rate vector (i?i, i?2, ■ ■ • , Rl) 
is {T,D^)-admissible if there exists a sequence {{ipi' , 



'P2 



(n) 



Vl\ V'^"^)}^=i such that 
limsupi?*") <Ru forleAL, 
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r ' ■k^x'<-x' 



kk 



< Dk, for keAK, 



where [C]ij stands for the (i, j) element of the matrix C. Let 
7^L(^,£)^|Ex/cyI.) denote the set of all (r,i:>^)-admissible 
rate vectors. When F is equal to the K x K identity ma- 
trix Ik, we omit F in 7?.L(r, D|E_5(-A-yL) to simply write 
TIl{D\Yixky'-)- Similar notations are used for other sets 
or quantities. The sum rate part of 7?,L(r, Z?^|Exr'yi.) is 
defined by 



-Rsum,L(r,-D \Tjx'<Y^ 
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Problem 2. Sum Distortion Criterion: Fix K x K positive 
definite invertible matrix F and positive D. For given F and D, 
the rate vector [Ri, R2,- ■ ■ , Rl) is (F, D)-admissible if there 
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exists a sequence {{ip^^' , f''^' , • • • , (yS^"'', '^'•"Oln^i such that 
limsup i?(") <Ri, for 1<E Al, 



lim sup tr 
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< D. 



The sum rate part of TZl(X,D\11xky^) is defined by 

,L(r,i:)|ExKyi.) = _ pin „ , S y^fl; 



Rs 



mm 

{Ri.R2.---,Rl) 

eiZL{r.D\s^^KYL) 



.1=1 



Let Sk{D^^) be a set of all K x K covariance matrices whose 
{k, k) element do not exceed Dk for k G Kk- Then we have 

7^L(^,I?^|Ex^yL)- y 7^L(s]d|Ex^Y^),(l) 

7^t(^,Z?|E;fKyL)- y 7^L(Erf|S;fKyO■ (2) 

tr[rSd'r]<D 
Furthermore, we have 

7iL(r,7^|E;fKv..)= y 7^L(^,i5^|I]^Ky.). (3) 

In this paper we establish explicit inner and outer bounds of 
'R.L{^d\^x'<Y^)- Using the above bounds and equations ^ 
and (|2]), we give new outer bounds of 7?,L(r, D\Yjxkyl) and 

B. Inner Bounds and Previous Results 

In this subsection we present inner bounds of TZLC^d 
l^x'^Y^), nL{T,D^ l^x^Y^), and TZl{T,D IS^ifyi,). 
Those inner bounds can be obtained by a standard technique 
developed in the field of multiterminal source coding. 

For / e Al, let Ui be a random variable taking values in the 
real line M. For any subset 5 C Al, we introduce the notation 
Us = iUi)ies- In particular Ua, - C/^ = (C/i, C/2, • • • , Ul). 
Define 

g{T,d) ^ {U^ : C/^ is a Gaussian 

random vector that satisfies 

IjL ^Y^ ^ X^ 
for any S* C A^, and 

for some linear mapping 

V' : M^ -^R^. } 

and set 

= conv {i?^ : There exists a random vector 
U^ e 5(Srf) such that 

Y,Ri>I{Us;Ys\Us^) 
les 
for any S C A^. } , 

where convjA} stands for the convex hull of the set A. Set 

= conv<| y 7^L(Ed|Sxityi) 

= COnV< y 7^L(Sd|Sx■^■Fi) 

I tr[rSd'r]<£' 



A 



Define 

and set 

d {VY,xK\Y^ r) = ([rSx^iFi- r]ii, [rsjc^ir^ r]22, 



[rs 



r]i 



_5(-K|yI, i JA'K 



We can show that 7?,^"^(Erf|SxKyi.), 7e^"'(r, D^lS^Kyi,), 
and iZj^ (T ^ D\Yixkyl) satisfy the following property. 
Property 1: 

a) The set TZ™ {Yid\^x'^Y^) is not void if and only if S^ >~ 



E^^lyL 



,(m)/ 



b) The set TlfiV, D^ \^x'<yI') is not void if and only if 

D^ > d^{T SxK|yI,'r). 

c) The set 7?,}J" (F, Z?|Ey^f^) is not void if and only if 

D > tr[ri;x/f|yi.*r]. 

On inner bounds of TZii^dl^x'^Y'-)^ 7?.L(r, -D^|Sx^'y^ 
), and 7^i(r, D|I]j5s:^y^)> we have the following result. 

Theorem 1 (Berger ^ and Tung US]/): For any E^ >- 
EjfK|yi,, we have 

7^f '(EdiExA-yO C 7^L(Ed|ExK-yL). 
For any T and any D^ > d^(rEjfR-|yL'r), we have 

7^<;"'(^,i?^'|E;tKyO c 7^L(^,i?^'|ExA-y^). 

For any F and any D > tr[FExJ^|yi*F], we have 

7^f ^(F,i?|E;,K-yO C 7eL(F,i?|ExKy.). 

The above three inner bounds can be regarded as variants 
of the inner bound which is well known as that of Berger m 
and Tung O. 

When A' = 1 and Lxl column vector A has the form A = 
*[11 • • • 1], the system considered here becomes the quadratic 
Gaussian CEO problem. This problem was first posed and 
investigated by Viswanathan and Berger fTSl . They further 
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Viswanathan and Berger ifTsl studied an asymptotic form of 
Rsuiw{D\a\ , a^) for small D. Subsequently, Oohama lfT6l 
determined an exact form of Rsun\{D\<7\ , a^) . The region 
TZl (-D|Exyi ) was determined independently by Oohama ifTTl 
and Prabhakaram et al. ifTsl . Wang et al. (T2\ obtained the 
same characterization of -Rsum.L {D\Y,xy'- ) as that of Oohama 
iflTl in a new alternative method. Their method is based 
on the order of the variances associated with the minimum 
mean square error (MMSE) estimation. Unhke the method of 
Oohama f[7\ . the method of Wang et al. lfT2l is not directly 
applicable to the characterization of the entire rate distortion 
region 7^I,(i:)|Exyi)■ 

In the case where K = L = 2 and T ^ A = I2, Wagner et 
al. ifTTI determined TZ2{D^\ Ex2y2). Their result is as follows. 



Theorem 2 (Wagner et al. jfTTV ); For any D'^ > (P{[i:x^\ 
yA)^ ^^ have 

Their method for the proof depends heavily on the specific 
property of L = 2. It is hard to generaUze it to the case of 
L>3. 

In the case where K = L and T = A ^ I^, Oohama 
||20l - ll24l derived inner and outer bounds of TZl{D\J^x'^y^)- 
Oohama II2TI . ||23l . 1241 also derived explicit sufficient condi- 
tions for inner and outer bounds to match. In 1221 . Oohama 
derived explicit outer bounds of TZL{^d I'^x^Y'-): T^l{D^ 
lExi-yi), and TZl{D iS^i-yi.)- 

The determination problem of TZl{D\T,xky^) in the case 
where A is a general K x L matrix and T = Ik was studied 
by Yang and Xiong 1261 and Yang et al. l27l . Relations 
between their results and our results of the present paper will 
be discussed in the next section. 

III. Main Results 
A. Inner and Outer Bounds of the Rate Distortion Region 

In this subsection we state our result on the characterizations 
of 7^L(Sd \^x'<Y^), nL{T,D^ iSx^y^), and 7^L(^,D 
\Yjxkyl). To describe those results we define several func- 
tions and sets. For each I e A^ and for ri > 0, let 
Ni{ri) be a Gaussian random variable with mean and 
variance aj^ /(I — e"'^'"'). We assume that iV;(r;), / G A^ are 
independent. When r; — 0, we formally think that the inverse 
value (J]^]q) of the variance of A'';(0) is zero. Let Ejv^(r^) be 
a covariance matrix of the random vector 

When rs = 0, we formally define 

Fix nonnegative vector r^. For 9 > and for S C A^, define 



b) Suppose that r^ G ALi^d)- If r^l^. ^q still belongs to 
ALi^d), then 



Jsi\^d\,rs\rs-)\rs=o 
= 0. 



Jsirs\rs-)\ 



rs=0 



Property 3: Fix r^ G ALi^d)- For S C A^, set 

fs = fsirslrs") = J_s{\^d\,rs\rs-). 

By definition, it is obvious that fs,S C A^ are nonnegative. 
We can show that / = {fs}scAL satisfies the followings: 

a) h = 0. 

b) Ia < Ib for a C S C Al. 

c) fA + Ib < Iahb + Iaub- 

In general (A^,,/) is called a co-polymatroid if the nonneg- 
ative function p on 2^'^ satisfies the above three properties. 
Similarly, we set 



fs = fs{rs\'rs'') = JsirsVs") , / = |/s| 



SCAi 



Then (A^,,/) also has the same three properties as those of 
( Al , /) and becomes a co-polymatroid. 
To describe our result on TZL{^d\'^x'^Y^)^ set 



,(out). 
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JsiO,rs\rs^) = ^log-' 
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Nsc (rsc ) 
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A 



where 5"^ = A^ — 5" and log~''[x] = maxjlogx, 0}. Set 



AL(^d) = \r'^>0 
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We can show that for 5 C A^, J^dE^j, rs\rs<^) and 
Js{f's\fS'') satisfy the following two properties. 
Property 2: 
a) If r^ G ALi'^d), then for any S C Al, 

J.si\^d\,rs\rs':) < Js{rs\rs-)- 



{i?^ : ^i?i> Js(0,rs|rsc) 
ies 
for any S" C Al. } , 

7^i°"')(E,|E;,Ay.) 

^ U nt'\\j:d\,r^\j:x^Y^] 

{i?^ : Y.^i^-^s{rs\rs^) 
les 
for any 5 C Al. } , 

= convi U 7^f)(r^|E;,Ay.) 

[r^eALi^d) 

We can show that 7^^"'(Ed|ExA'YL) and 7^^°"'^(Ed|ExicyI,) 
satisfy the following property. 

Property 4: The sets 7^^'"'(Ed|ExA'yi) and 7^^°"'^(Ed 
jEx^yi-) are not void if and only if E^ >- I]x'<\y'-- 

Our result on inner and outer bounds of TZLi^dl^x'^Yi-) 
is as follows. 

Theorem 3: For any E^)^ E^xiyt, we have 

C 7^L(Srf|ExifYi.) C 7^^ (Ed|Exifyi). 

Proof of this theorem is given in Section V. This result 
includes the result of Oohama ll22l as a special case by letting 
K ^ L and T = A = I^. From this theorem we can 



derive outer and inner bounds of TZl{T,D^\ 'Ex'^y'^) ^"d 
7?.i,(r,Z?|I]j(-/<yi,). To describe those bounds, set 



A 



A 



convi U 7^f)(Ed|S^K 



yL , 



trfrSd'ri<_D 



<'"'(r,i?|I]xA-y.) 



Set 



= conv<^ U 7^^(S,|S^K^.) 



0(r,i?^,r^)= max |Ed| , 

rEd*re5ic(D^) 
0(r,Ar^)= max |E<i|. 

EdiEieytiCr-f-), 
tr[rEd*r]<D 

Furthermore, set 

{r^ > : r(E-i. + MS-i(^,jA)-"r G SKiD")] , 

Bl{T,D) 

{r^ > : tr[r(E-i + *AE-i(^,^A)-"r] < i^} . 

It can easily be verified that 7^^°"''(^, D^'lE^KyL), 7^^"^( 

r, D^\LxKY^), 7^^™''(^, d|Ex^fO^ and 7^^"^(^, d\ 

EjfKyi) satisfies the following property. 
Property 5: 
a) The sets 7^i'"^ (r,D^|Ejf icyi,) and 7^^°"*^(^, D^IE^k 



A 



A 



n^t\^^D\Ex'<Y-) 

-convJ U 7^f'(r^) 

[rieZ3i.(r,D) 

The following result is obtained as a simple corollary from 
Theorem [3] 

Corollary 1: For any T and any D^ > d^(rEx/f i^i-'r), 
we have 

Tef )(r,i5^'|E^,cy.) = 7^f ^r,i?^|E^Ky.) 
c 7^i(^,i?^|E^Ky.) c 7^(°"''(^,i?^|E^Ky.). 

For any F and any D > trpEj^fA-iyt'r], we have 

nP{T,D\j:x'<Y-) = nl'"\T,D\j:xxY^) 
c 7^i(^,i?|ExK■y.) c 7^i°""(^,i?|ExKy.). 



Those result includes the result of Oohama 1221 as a special 
case by letting K = L and T = A = I^. Next we compute 
9{r,D,r^) to derive a more explicit expression of 7?.]°" (F 
, D\TjxKYi- )• This expression will be quite useful for finding a 
sufficient condition for the outer bound 7?.^" (F , Z3|E y^y t ) 
to be tight. Let ak = ak{r^),k G Ak be K eigenvalues of 
the matrix 



Y^yK + AY.^L(^L^A] F 



^XK ' ''-^N^(r^y 

Let ^ be a nonnegative number that satisfy 

K 



E{K-"^-']^ + "^"'} = ^- 



fc=l 



Define 



{T,D,r^)^\Tr'[{{[i-air + 0^1'] 



k=l 



The function cj(F,D,r^) has an expression of the so-called 
water filling solution to the following optimization problem: 



K 



c^(F,i?,r^) = |F|- 



max 

{fcQfc>i,fceAK- , 






(4) 



yl) are not void if and only if D^ > d^(FExK|yi,'F). 
b) The sets 7^^"^(F,D|ExKyL) and 7^^™*^(F,D|ExK y^) 



c) 



are not void if and only if I? > tr[FExA-|yt 'F]. 

n'^°-^\T,D^\j:x'<Yr.) 

U 4°"*)(0(F,i?^,r^),r^|E^.y.), 

7ef)(F,i?^'|Ex-y.) 

convJ U 7^f'(r^|E;,Ky.) 

7^i°"')(F,i?|Ex-y.) 

U <™''(0(F,Ar^),r^|E^Ky.), 

r^eBLir.D) 



Then we have the following theorem. 

Theorem 4: For any F and any positive D, we have 

0(F,i?,r^)=c^(F,i?,r^). 

A more explicit expression of Tcf^ {T,D\'Sx'^y'-) using 
Ci;(F, D, r^) is given by 

<°"')(F,i?|E;,Ky.) 

^ U 7ei™*)(^(r,i?,r^),r^|E^Ky.). 

Proof of this theorem will be given in Section V. The above 
expression of the outer bound includes the result of Oohama 
1221 as a special case by letting K = L and T ^ A ~ I^. 
In the next subsection we derive a matching condition for 

7e[°"*^(F,D|E,YAYL) to coincide with 7^L(F, D|EjfKyi,). 



Two other outer bounds of 7?.L(£'|S;if/cyL) were obtained 
by Yang and Xiong 1261 . They derived the first outer bound for 
general Lx K matrix A. This outer bound denoted by 7?.)°" ( 
D|SxA'yi) does not coincide with 7?,]°" ( D\T,xky'^) when 
K = L and A = I^. When A is semi orthogonal, i.e., 
^AA = Ik, Yang and Xiong 1261 derived the second outer 
bound n^°'''\D\T,xKYL) tighter than 'TI^°'''\d\T,xi<y^)- 
The outer bound 7^]°" (-DjEx^yj^) is the same as our outer 
bound 7?.]°" ( D\Y,xkyl) although it has a form differ- 
ent from that of our outer bound. They further derived a 
matching condition for ^]°" '{D\T,x'^y'^) to coincide with 
TZl{D\Y,xky^)- Their matching condition and its relation 
to our matching condition will be presented in the next 
subsection. 



B. Matching Condition Analysis 

For L > 3, we present a sufficient condition for 7?.}°" (F, 
D\ E^^A-yi,) C 7?.)J' (I?|I]x«'y^)- We consider the following 
condition on 0(F, D, r^). 

Condition: For any / e A^, c~^^''9(r, D, r^) is a monotone 
decreasing function of r; > 0. 

We call this condition the MD condition. The following is 
a key lemma to derive the matching condition. This lemma is 
due to Oohama 1211. l23l. 

Lemma 1 (Oohama l[2Tll.l[23]l): If 9{T,D,r^) satisfies the 
MD condition on Bl{T,D), then 

n^t\T,D\^x^Y^) = nL{T,D\j:x^Y^) 

^TZ'^r'\T,D\J:x-Y-)- 

Based on Lemma [T] we derive a sufficient condition for 
9{T,D,r^) to satisfy the MD condition. 

Let aik be the (/, k) element of A. Set a; = [aiiai2 ■ ■ ■ uik] 
and d; — ajF"^. Let Ok be the set of all K x K orthogonal 
matrices. For (/, fc) e A/, x Ak, let 0/<-(d;, fc) be a set of all 
T e Ok that satisfy 



[aiT], = 



|d(||, if j = fc, 
0, if j ^ k. 



For T S Ok{(1i, k), we consider the following matrix: 

L 



1=1 



A 



Let rh,=ri--- n_in+i • • • r^ and set 



'W 



Vk{T~^T,r 



[I]' 



A 



[*T*r-ii]^ir-iT]^^ 






-2ri 



A 



x;,(r-ir,,f])^||d,|p^ 



nd«T)(a,T)],,, 



%(r-ir,r^ 



Then we have 

[C(r-iT,r^)]fcfc 



|2 1 



If (i',i") 7^ (fc,fc), then the value of 

[C(r-ir,r^)],,,. 



2'-o + r/.(r-ir,r^,) 



P^C-2r,^ 



(5) 






-2ri 



Hd,r)(d,r)]^,^ 



does not depend on r;. Note that the matrix C(r ^T,r^) has 
the same eigenvalue set as that of 

We recall here that ak = afe(r^), k e Aa' are K eigenvalues 
of the above two matrices. Let amin = ciniin('''^) and ckmax = 
amax(?'^) be the minimum and maximum eigenvalues among 
afc,fc e Aa'. The matrix C(r^ir,r^) for Te OA(a/, fc), has 
a structure that the (/c, fc) element of this matrix is only one 
element which depends on r; and this element is a monotone 
increasing function of r; > 0. Properties on eigenvalues of 
matrices having the above structure were studied in detail by 
Oohama l2T]| . l23l . The following lemma is a variant of his 
result. 

Lemma 2 (Oohama 427l/ .^25l/).- For each (/, k) e A^, x Aa 
and each T e Ok{0'U k), we have the followings. 



< \\ai 

daj_ 
dr. 



12 1 



(1 



''')+mk{T-^T,rfi^)<a^Ur'^), 



K 



> 0, for j e Aa, J2 



j=i 



dri 



2\\ai\ 
p2n ^2 



N, 



The following is a key lemma to derive a sufficient condition 
for the MD condition to hold. 

Lemma 3: If anuni^^) and amax(''^) satisfy 



1 



1 



amin(r^) 
for I e Aa, 



.(r^) 



c(r^) ^ e-'cr 



2n^2 

N, 



,(r^) 



\ai\ 



(6) 



on SL(r, £>), then 9{r, D, r^) satisfies the MD condition on 
Bl{T,D). 

Proof of Lemma [3] will be stated in Section V. Set 



A 



c*(r-ir,r()= lim c(r-ir,r^), 



XI{T-'T)^ lim xik{T-'T,rfi^) 

= [tr*r-i (E-4 + *Ai]^i A) p-^T] 



fcfc- 



For k e Aa", we denote the (fc, /c) element of C* (F ^T,ri) by 
cL = 4fe(r-^T,n). When U,f) e A^, and 0-,j') ^ (fc, fc), 
the (j, j') element of C*(r ^T,ri) does not depend on rj. 
We denote it by c*j, — c*^, (F^^T). Furthermore, set 



'fc[fc] 



<w(r-^T)^[4i. 



^kk-l'--kk+l ' 



■ CfcA"]- 



By definition we have 



where 



cl,{T-'T,r0^xU^-'T) 



\(^i\ 



=^-ria^. 



^ A J 1 [aiT]l 

T ; — max max ' 



Ni 



Define 



fioK fee Ak [xliT) \\aiT\\^ ^ 
The matching condition (fTTT i by Yang et al. ||27| also improves 



= lim a,nax('''^),amm = lim amin(?'^), 



that of Oohama ||23J,||24J in the case of K = L, A = II- When 
r = /x, for / G Ai, we have 



"max(''i) = 1™ amax(?"^) for I ^ Al. 

By definition, a* „^ and a*- are the maximum and minimum 
eigenvalues of ^r~'^{T,^\ + 'AS^i A )r"\ respectively. By 
Lemma 121 we have 



T, = 



1 



max 

keAfc * /rp\ 

TeOK(ai,k) Xk\-^ ) 



|C^W(T)||^ 



^wCTIP 



(Xndnir^) < amin(n) < ttmin. for ' € A^, 



(7) 



. 1 A 

> max — -— = 1 

- feeAA- xliT) - 

TeOK{ai,k) 



(12) 



Xik{r-^T, rfi^) < Xfe(r-ir) < «,*„,,, for / e Al. (8) On the other hand, for ^ e Al, we have 

1 [o-iT]t 



The following lemma provides an effective lower bound of 

Lemma 4: For any (/, fc) e A/, x A^ and T e Oyiai, k), 
we have 

cUT-^T,ri)=xl{V-^T) 



m\ 






> amin(n) + 



<iax(»"0 - amin('"0 

"4[fe](r-^T)IP 



Proof of this lemma will be given in Section V. Set 



T,(r-i) 



A 



max 



1 + "''^- 


w(r- 


^T)||' 






(";,a. 


)■' 




x^(r-ir)- 


llq, 


.l(r- 


1T)||2 




f^i^^^ 





When r = /x, we simply write Ti{Ik) = T;. From Lemmas 
[T]|4]and an elementary computation we obtain the following. 
Theorem 5: If we have 



tr[rExA-|yi.'r] <D < + mill Ti{T-^) (9) 



a' 



eAi 



then 



>(in)/ 



= 7^i(^, i?|SxKy. ) = 7^i°"" (r, t^is^^ xy. )■ 

Using (O, we obtain T((r^^) > \/a*^^^. Hence we have the 
following matching condition simpler than (|9]): 

K+\ 



trpEx-f^iy^ r] <D < 



a 



(10) 



Proof of Theorem |5] will be stated in Section V. When 
K = L^A = II, the matching condition (fTOl i is the same 
as that of Oohama Il23l . ll24l . It is obvious that in the case of 
K = L,A = II, the matching condition (|9]l improves that of 
Oohama 1231 . 1241 . Yang et al. [271 have obtained a matching 
condition on TZl{D\J^x'^y'-) by an argument quite similar to 
that of Oohama fJSl. The matching condition by Yang et al. 
EJ\ is as follows: 

K 

ti[ExK\Y^] < D < hminT/, (11) 



T; = max max 



TeOxfcGAK {xliT)\\aiT\\^^ 
1 [aiT]l 



max max 



fe6AK-TGOK IxliT) ||azT||2 

m.ax max < — 7T:;T--r, — ;;^ 
keAKT&OKia,,k) [XkiT) ||a/T||2 

1 



(13) 



max max 



keAKTeOK{a,,k) xl{T) 



X/- 



Thus, we have T;> Tj and T;> T;. Comparing the two 
inequalities (fT2l i and ( fTSl l. we can see that the improvement 
of T; from T; is quite differnt from that of T; from T;. Hence 
we have no obvious superiority of T/ or T/ to the other. 

Next we derive another matching condition, which is better 
than the second matching condition ( fTOl i in Theorem|5]and the 
matching condition (fTTT l of Yang et al. l27l for some nontrivial 
cases. Set 

A cr^r, * A . 

'''I ^ w- 119 ' ''' = inin '''' • 

From Lemmas [Tl[3] and an elementary computation we obtain 
the following. 

Theorem 6: If we have 



<D< 



tr[ri;xK|y-c*r] 

K 1 



V* Ort* 

max ^^inax 



1 + 4a*,,,r* 



l}, (14) 



then 



Proof of Theorem |6] will be stated in Section V. When r* 
becomes large, a*jax ^^'^ "^min approach to the maximum and 
minimum eigenvalues of S^^, respectively. Hence we have 

;ini -^ I VI + 4ar„ax^* - l} = +00, (15) 

r--.+oo 2a*,^^ L ^ J 

which implies that there exists a sufficiently large r* such that 



1 1 
< ^— < 



2tf^l^ 



l+4<axT*-l}. (16) 



iVi 



Xi 



N2 



X. 



Nu 



Xl 



v^. 




^'r\Yi) 


Yi 


^ i 


(") 







Let Y, j^_^L be a covariance matrix with du> in its {1,1') 
element. 

For a given S^, the rate vector {Ri, R2,- ■ ■ ,Rl) is S^- 
admissible if there exists a sequence {((^![" , 1^92 1 ' ' ' 1 Vl ^ 
-0^"^)}^=! such that 



v^ 




^r(^2) V 




>"2 


-» 2 


(n) 


0(n) 




/ 



1^2 






limsupi?^^ < i?;, foxle Kl, 

n—^oo 



Yl 






vt\yL), 



Fig. 2. Distributed source coding system for L correlated Gaussian sources 

On the other hand, it follows from the definition of T; that 
we have for / e A^, 



Let 7?.L(Sd|Syi.) denote the set of all S^^-admissible rate 
vectors. We consider two types of distortion criterion. For each 
distortion criterion we define the determination problem of the 
rate distortion region. 

Problem 3. Vector Distortion Criterion: For given L x L 
invertible matrix F and D^ > 0, the rate vector (i?i , R2 , 
■■■,Rl) is (T, D^)-admissible if there exists a sequence 
{(^("\^("\...,^i"\ </-("))}-! such that 

limsupi?'"' <Ri, for le Al, 



T; < max max 



1 



< 



1 



(17) 



TeO^keAKXliT) ^in' 

Thus we can see from ( fTSI l and ([TtI i that for sufficiently large 
T*, the matching condition ( fT4l i in Theorem |6] is better than 
the second matching condition (fTOl i in Theorem |5] and the 
matching condition ( fTTT i of Yang et al. ||27l . 

IV. Application to the Multiterminal source 

CODING PROBLEM 

In this section we consider the case where K = L and 
A = II. In this case we have y^ = A"^ + N^; Gaussian 
random variables Yi, I G Al are L-noisy components of 
the Gaussian random vector AT^. We study the multitermi- 
nal source coding problem for the Gaussian observations 
Yi,l G A. The random vector A'^ can be regarded as a 
"hidden" information source of Y^. Note that (X^,y^) 
satisfies Ys -^ X^ -^ I5C for any 5 C A^. 

A. Problem Formulation and Previous Results 

The distributed source coding system for L correlated 
Gaussian source treated here is shown in Fig. |2] Definitions 
of encoder functions (pi,l G Al are the same as the previous 
definitions. The decoder function c/)'"^ is defined by 



lim sup 



F — S., r v>J 



<Di, for le Al. 



Let 7^I,(F,i:)^|EyI,) denote the set of all (F,i:)^)-admissible 
rate vectors. The sum rate part of the rate distortion region is 
defined by 



^sum,L(r,-D ISyi.) 



A 



mm 

eTCi,(r,_D^|Syi) 




Problem 4. Sum Distortion Criterion: For given LxL invert- 
ible matrix F and D > 0, the rate vector (i?i, i?2, ■ ■ ■ > Rl) is 
(F, D)-admissible if there exists a sequence {(^Pi , ^2 7 ' ' ' ; 
vi"\ 0*"^)}^=i such that 

limsupi?'"' <Ri, for / e Al, 



lim sup tr 



F I — X;-,r _v^ 



tp 



< D. 



Let 7?.L(F,_D|Syi,) denote the set of all admissible rate 
vectors. The sum rate part of the rate distortion region is 
defined by 



A 



-Rsum,L(r,£'|i;yl,) = mill 

(Ri,R2,---,Rl) 



.1=1 



4"\4"\' 



■,0i"^) 



a(") 



:7Wi X ••• X A/(i h^R",; e Al. 



ForY^ =(1^1, Y2, •••,yL), set 



Yl 
Y2 



du^n\Yi 



A 



4«)(^(«)(yi)) 



A 



^{Yi-YuYi,-Yi,),l<l^l'<L. 



Relations between 7^L(I;d|SYL), 7^L(F, D^lSyi,), and 
7?,L(r, D|EyL) are as follows. 

nL{T,D'^\LyL)^ U 7eL(Ed|S]y^), (18) 

rSdtre5i(£i^) 

7^L(^,i?|Ey.)= U 7^L(Ed|Ey.). 

tr[rSd'r]<_D 
Furthermore, we have 

7^L(^,i?|s]yi)= U 7eL(r,i?^|Sy^). 

We first present inner bounds of 72.^(1]^ |Syi.), 7?.l(F, Z?^ 
|Eyi,), and 7?.l(F, Z?|I]yi). Those inner bounds can be 



(19) 



(20) 



10 



obtained by a standard technique of multiterminal source 
coding. Define 



A 



Gi^d) = {U 



U^ is a Gaussian 

random vector that satisfies 

U^ ^Y^ ^ X^ 
for any S C A^ and 

for some Unear mapping 

0: R^^R^. } 



and set 



7^r^(I],|I]^.) 



A 

= conv 



|i?^ : There exists a random vector 
U^ e ^(Sd) such that 
Y,Ri>HUs;Ys\Us^) 

for any S C A^. } , 



7^f)(^,i?^|Ey.) 



A 

= conv ■ 



i(in) 



u 



7^f)(Srf|Ey.) 



rSd'Te5i,(Di 



7^^(^,i?|Sy.) 



A 

= conv 



u 



7^f)(Srf|Sy.) 



trfrEd'rl<L> 



Then we have the following result. 

Theorem 7 (Berger ^ and Tung 15]): For any positive 
definite S^, we have 

7^f^(Ed|Ey.)C7^L(Sd|Ey.). 

For any invertible F and any D^ > 0, we have 

For any invertible F and any 13 > 0, we have 

'r!-^^\v,d\Lyl) c7ei(F,i:»|EyL). 

The inner bound 7^}J"^(D^|Eyi,) for F = /^ is well known 
as the inner bound of Berger ID and Tung ||5|. The above three 
inner bounds are variants of this inner bound. 

Optimality of 7^2 (^^l^y^) was first studied by Oohama 
JS). Let 



Theorem 8 (Oohama ^j." For / = 1, 2, we have 

7^^,2(A|Ey2)=7^r.2(A|Ey2), 

where 

7^;*2(A|Ey.) = 

[{R1.R2) -.Ri > ilog+ [(1 -P')^ (1 + T^ ■ 

for some < s < 1 > . 

Since 7?,;*2(£'i|Ey2), / = 1,2 serve as outer bounds of 
7^2(£'^|Ey2), we have 

7^2(£'^|Ey2) C 7^l_2(^l|Sy2) n 7^2.2(£'2|Ey2). (21) 

Wagner et al. ifTTI derived the condition where the outer bound 
in the right hand side of jTH is tight. To describe their result 

set 

V={{Di,D2):Di,D2>0, 

max{^,f}<min{l,p2min{^,f} + l-p2}}. 
Wagner et al. IfTTI showed that if D^ ^ T>, we have 

n2{D^\^Y2) = ni2{Di\Y.Y2) n 7^;_2(£'2|Ey2). 

Next we consider the case of D^ E T). In this case by an 
elementary computation we can show that 7^2 (-C^|Ey2) has 
the following form: 

= 7^*,2(£'l|Sy2) n 7^;_2(^2|Ey2) n ni2{D^\^Y-) , 

where 

7^^,2(^'|SF2) 



A 



[{RuR2) 



Rl+ R2> Rslm,2 



(D'l^Y^)}, 



?(u) 



i?Z,2(i?lEy2) 



A 



(iil,-R2)eK^'"'(D2|S^2) 



{i?l + R2} 



ilog 



1-p' 



D1D2 



-D1-D2 



4p'^ 



(l-p2)2 



Ey2 = 

For / = 1,2, set 



CTi 



p(Jl(J2 



paia2 0-2 



pe[0,l). 



7^^,2(A|Ey2) = U 7^2(Z32|Sy2). 

Oohama ||9] obtained the following result. 



The boundary of 7^2 (^^ I ^y^) consists of one straight line 
segment defined by the boundary of TZ'^ 2(^^l^y^) ^^'^ ^^o 
curved portions defined by the boundaries of TZl 2(^i|Ey2) 
and 72.22(^21^^2). Accordingly, the inner bound estab- 
lished by Berger f?! and Tung 151 partially coincides with 
7?.2 {D^ I Ey2 ) at two curved portions of its boundary. 

Wagner et al. IfTTI have completed the proof of the opti- 
mality of 7^.2 (.D^|Ey2) by determining the sum rate part 
.Rsum.2(£'^|Ey2). Their result is as follows. 

Theorem 9 (Wagner et al. UTTH]: For any D^ E V, we have 

Rsun.,2{D'\^Y2) = Ri'^l.2{D'\^Y2) 



ilog 



D1D2 



\D1D2j 



ip^ 



(l-p2)2 
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According to Wagner et al. IfTTI . the results of Oohama 
|fT6l . ifTTl play an essential role in deriving their result. Their 
method for the proof depends heavily on the specific property 
of L = 2. It is hard to generaUze it to the case of i > 3. 
Recently, Wang et al. lfT2l have given an alternative proof 
of Theorem |9] Their method of the proof is quite different 
from the previous method employed by Oohama |[T6| . IfTTl 
and Wagner et al. fTTI and also has a great advantage that it 
is also applicable to the characterization of Rs-am.L{D^\^Y^) 
for L > 3. Their result and its relation to our result in the 
present paper will be discussed in the next subsection. 

B. New Outer Bounds of Positive Semidefinite Programming 

In this subsection we state our results on the characteriza- 
tions of 7^L(Srf|SyI,), 7^L(^,i:>^|SyL), and 7^L(^, Z^lEyi,). 
Before describing those results we derive an important relation 
between remote source coding problem and multiterminal 
source coding problem. We first observe that by an elementary 
computation we have 

X^ = AY^ + N^ , 



From 



we have 



is. 



= A 



-1 / 1 



-S, 



,.L\'A'^ -B 



^ ^ ' n^X^-X' 



A- 



(27) 



Conversely, we fix {{ip\'' , (p2 \ 



(n) 



("))}-!, arbi- 



trarily. For each n = 1, 2, • • •, using the estimation Y of Y 
given by 



- L 

Y = 



where A 



(S7^ 



(22) 



-E„i) ^S„i and N^ is a zero mean 



Gaussian random vector with covariance matrix E 



JV-L — (S^-t 



-ATI 



^. The random vector N^ is independent of F^. Set 



A 7 



B = A~^Y.^r.^A-^ =I]«i, +S 



■•N 



-•N^'^X^^N^ 



6^^* 



B = TB^T. 



([i?]n,[i3]22,---,[S]LL) 



h'^=\[BU,[B]22,---,[B]LL). 

From ( |22] |. we have the following relation between X and 



X' 



AY' 



n\ 



(23) 



^^L 



where N is a sequence of n independent copies of N and 
is independent of Y . Now, we fix {{ipi, '^2 \ ' ' ' 1 yl^ 
-0^"^)}5^]^, arbitrarily. For each n = 1,2, • • •, the estimation 
X of X is given by 



X 



0(")(^(")(F^)) 
4"^(^(")(F^)) 

0i")(^(")(Y^)) 

we construct an estimation X of X by ( l24l i. Then using 
(|23] l and ( |24] |. we obtain (|25l l. Hence we have the relation ( [26] l. 

The following proposition provides an important strong 
connection between remote source coding problem and mul- 
titerminal source coding problem. 

Proposition 1: For any positive definite E^, we have 

For any invertible F and any Z?^ > 0, we have 

'Rl{T,D^\^y^) =1^l{TA-\D^ + h^\^x^Y'^)- 
For any invertible F and any Z? > 0, we have 

7^i(^,I?|I]yL) = 7^L(^i-^I? + tr[s]|Ex^y^). 

Proof: Suppose that R^ e 7^L(i(Ed + B)*i|E^i,yL). 
Then there exists {((p]^" , ip2\ ■ ■ ■ , <Pl , ^'"^)}J5^i such that 

limsupi?'"' <Ru for I e Al, 

?l— >-oo 

limsup i^x^_x^ ^ M^d + BY A. 

n— f 00 
- L ' L T - L 

Using X , we construct an estimation Y of Y by y = 
A^^X . Then from (|27] |. we have 

limsup iEyi_^i. 

n— >-CXD 



= limsup A-MAE^ 

n— ^00 ^ 



I'l^ ^^ri-i-B 



- L 



Using this estimation, we construct an estimation Y of Y 
by V = A X , which is equivalent to 



X ^ AY 
From ( |23] l and JTM . we have 



X"^ -X 



rL 



Y ) + N' 



(24) 



(25) 



^ i"^i(E<i + S)'i*i"i - B = Erf , 
which implies that i?^ € 7^L(A(Ed + B)*i|Exi.yi.). Thus 

7^L(Erf|EyO D 7eL(i(Ed + B)*i|E;fZ,y^) 



Since 1^ is a function of Y , Y Y is independent of 
N . Based on (|25T l, we compute ^E^r ^^l to obtain 



is proved. Next we prove the reverse inclusion. Suppose that 

R^ e 7eL(Erf|Eyi,). Then there exists {i(p["\ 'pi"\---, ¥'i"\ 
0^"^)}^=i such that 



iE 



n X^ 



A 



n Y'- 



X^-X 

'i + E 



JV-C-- 



(26) 



limsupi?^") <Ri, for Z e Al, 



limsupiE.^j,_.^t ^ Eq 

n— ^00 
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- L - L T - L 

Using Y , we construct an estimation X of X by X = 
AY . Then from ( l26b . we have 

limsupiS^^^^L 



,VL I '^ + Sjvi 



= limsupAf iEyi .^^ 
which implies that R^ e 7^L(A(I]rf + B)'i |Sxi.yi.). Thus, 

7^L(Sd|I]yL)c7^L(i(Sd + B)*i|Sxiy^) 

is proved. Next we prove the second equality. We have the 
following chain of equalities: 

rSd'reSt(D^) 
rEdtre5t(D^) 

-rB'rG5L(-D^) 



rA"M(Ed+s)M'(rA"^) 



u 



7?.L(Sd|Sxiyi-) 



= 7^L(^i-^L'^ + 5^|S]xi,yi,). 

Thus the second equality is proved. Finally we prove the third 
equality. We have the following chain of equalities; 

tr[rSd<r]<_D 

y 7^i(^i(Srf + B)*i|S;,.y.) 

tr[rSrf'r]<D 

tr[rA-iA(Sd + B)'A'A""r] 
-tr[rB'r]<I3 

<Ll+tr[B] 

= 7^L(^i-^i? + tr[B]|Ex^y^). 

Thus the third equality is proved. ■ 

Proposition[T]implies that all results on the rate distortion re- 
gions for the remote source coding problems can be converted 
into those on the multiterminal source coding problems. In the 
following we derive inner and outer bounds of TZL{^d\^Y'-)^ 
7^I,(^,D^|EyI,), and 7^L(^,i:)|S]yL) using Proposition [T] 
We first derive inner and outer bounds of 7^i(I]rf|I]yi,). For 
each I G Ai and for r; > 0, let Vi{ri), I £ A/, be a Gaussian 
random variable with mean and variance aj^ /(c^"^' ~ !)■ We 



assume that V; (?■;), / e A^ are independent. When r; = 0, we 
formally think that the inverse value cr^ |q-, of Vi (0) is zero. Let 
SyLj^t) be a covariance matrix of the random vector V^^(r^). 
WTien rs = 0, we formally define 

y-l ^ y-1 

Fix nonnegative vector r^. For 9 > and for S C A^, define 

L 



ls{0,rs\rs^) ^llog+ 



Js{rs\rs<') = -log 




Set 



^L(Srf)= r^>0 



--1 I y-i 



^s. 



Define four regions by 

for any S C Al-} , 

TZPir^l^y.) ^ {R^ : Y^Ri > Jsirs\rs^) 

les 
for any S C Al-} , 

7^f ) (Ed I Sy . ) = conv J y -R^ (r^ | Ey . ) 

The functions and sets defined above have properties shown 
in the following. 
Property 6: 

a) For any positive definite E^, ^(E^) = ^(i(E<i + B)*i). 

b) For any positive definite Ec(, we have 

7^i;")(Erf|Eyz,) = 7^i;"^(i(Erf + i?)'i|Ex-y.). 

c) For any positive definite E^ and any 5* C A^, we have 

Js(|Ed + S|, rslrsc) = Js(|i(Ed + BYA\,rs\rs^), 
Jsirslrs") = Js{rs\rs-)- 

d) For any positive definite E^, AL{^d) = -^Li^i^d + 
BfA). 

e) For any positive definite E^, we have 

<°"*)(E,|E^.) = <°"')(i(Erf + B)'A\j:^.y.), 

7^1;"'(Erf|Ey.) = 7^i;"'(i(Ed + i?)*i|Ex^y-). 

From Theorem |3] Proposition [T] and Property |6] we have 
the following. 



Theorem 10: For any positive definite E^, we have 

c7^i(s]d|Si..)c7^(°""(I]<,|s]y.). 

Next, we derive inner and outer bounds of TZl (F,!?^^ jSyi, ) 
andTZLiT,D\Y.YL). Set 



AUr"^) = {Ed : Sd ^ (S 



yt 



^yt(ri) 



rl-)) I ■ 






A 



max I Erf 

rEd're5t(D^) 

max jErf - 

tr[rSd'T]<D 



SI 



B\ 



Furthermore, set 

BL{r,D) 



A 



= Ir^ > : tr 



<D 



Define four regions by 

;('n)i'T^ nil 



A 



A 



U 

-eBi,(r,_Di- 



7^f'(r^|Eyz,) 



,(out). 



= U 7^^*)(0>,i?,r^),r^|Ey.), 

r-f-eBiir,!)) 



jyt^ 



A 



conv [J 7^f\r^|Ey. 

[r-f'GBi.(r,_D) 

It can easily be verified that the functions and sets defined 
above have the properties shown in the following. 
Property 7: 

a) For any invertible F and any Z)^ > 0, we have 

7tf)(F,2?^|Eyz,) 

For any invertible F and any Z) > 0, we have 

7^f)(^,i?|Sy.) 

= n^i''HTA-\ D + tT[B]\Y.xLYL). 

b) For any r^ > 0, we have 

Srf e i(r^) ^ i(Sd + BfA e ^(r^), 



6i(F,L»^,r^) 



A 



' e{TA-'^,D^,r^), 



0{r,D,r^)^ A e{TA-'^,D,r^). 



c) For any invertible F and any D > 0, we have 

TZ^°^^\t,D^\^y^) 
= n^°'''\TA^\D^ + b^l^xLYL), 

>(m) 



7^^(^,i?^|Sy.) 
n^l^\TA-\D^ + b^\j:x^Y^)- 



For any invertible F and any Z) > 0, we have 

7^f)(^,i?|Ey.) 

= 7e^"^(ri-\ D + tr[i3]|E^i,yi,). 

From Corollary [1] Proposition [1] and Property [T] we have 
the following theorem. 

Theorem 11: For any invertible F and any Z) > 0, we have 

7e^"^(F,I?^|EyL)=7lf^(r,Z?^|S]y^) 
C7eL(F,L»^|SyL) C7^^°"''(F,L>^|S]yI,). 

For any invertible F and any 13 > 0, we have 

7^f'(F,7^|Sy.) = 7^f)(^,i?|Ey.) 

C7^L(F,L)|EyL) C7^^°"''(F,£)|SyI,). 

The outer bound 7^)°" '{T,D^\ Y^y^^ ) has a form of positive 
semidefinite programming. To find a matching condition for 
inner and outer bounds to match, we must examine a property 
of the solution to this positive semidefinite programming. On 
the sum rate part of the rate distortion region in the case 
of vector distortion criterion we have the following corollary 
from Theorem [TT] 

Corollary 2: For any D^ > 0, we have 



?(u) 



<ri::'l{d-\^y^), 



where 



?(u) 



= min -log|/ + EyiS-i,^,| 



mill 



1 |Sy.| 









mm 



log hZ ^ + 2^ r, 



\^d + B\ 



1=1 



A 



mm 



1 |SyL+B| ^ 

n log I J . J +ZZ' 



|Srf + S| 






! = 1 



s.t(s;;L+sr,L .,^-' 
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A lower bound of i?sum,L(£'^|Syi,) in a form of positive 
semidefinite programming was first obtained by Wang et al. 
lfT2l . Their lower bound denoted by W^^^^ ^{D^\Tiyl) is as 

follows. Let 5^ = ((5i, (52, ■ ■ ■ 7 ^l) be a positive vector whose 
components 6i, I G A^ belong to (0, cr^]. Let Diag.(i5'^) be 
a diagonal matrix whose {l^l) element is Si^l S A^,. Then 
^™m,L(^^|SyOisgivenby 



^iL,L(^''|SyO 



A 



1 



■N, 



= mm < - log — ; — h > - log ■ 

(S-i + B-i)-i>:Diag.(5^) 

By simple computation we can show that -Rg^jj, ^ (I?^ | Syi, ) 

= ^lum.Ll^'^lSyO- Although the lower bound R^l^j^{ 
D^jEyt) of Wang et al. lfT2l is equal to our lower bound 
R^LA ^^-iSy.), their method to derive i?^l J D^\^y^) 
is essentially different from our method. They derived the 
lower bound by utilizing the semidefinite partial order of 
the covariance matrices associated with MMSE estimation. 
Unlike our method, the method of Wang et al. is not directly 
applicable to the characterization of the entire rate distortion 
region. 

When L = 2, Wang et al. lfT2l solved the positive semidef- 
inite programming describing R\^^^ 2^ £'^|Ey2) to obtain the 
following result. 

Lemma 5 (Wang et al. U2f): For any covariance matrix 
Ey2, there exist a pair (S.y2, 11 m'^) of covariance and diagonal 
covariance matrices such that Sy2 = S^^ + ^m and 

^lL,2p'|SyO = i?luL(^'|SyO. 

From Corollary |2] and Lemma |5j we have the following 
corollary. 
Corollary 3: 



R. 



(1) 



i?su,„,2p'|SyO = i?IuL2(^'|Sy^)- 



Our method to derive i?£,^„^ 2 (^^ 1 5^yO < i?sum,2(-D^|Sy2) 
in Corollary |2] essentially differs from the method of Wang et 

al. IH to derive i?^!,^^ jl^'l^vO < Rsum.AD^l^Y^)- Our 
method to obtain Corollary |3] is also quite different from that 
of Wagner et al. |11| to prove Theorem |9] Hence, Corollary 
|3] provides the second alternative proof of Theorem |9] 



C. Matching Condition Analysis 

In this subsection, we derive a matching condition for 



,(out) 



(in). 



n'^ '(T^DIY^yl) to coincide with TZl '{T,D\J:y^)- Us- 
ing the derived matching condition we derive more explicit 
matching condition when F is a positive semidefinite diagonal 
matrix. Furthermore we apply this result to the analysis of 
matching condition in the case of vector distortion criterion. 
By the third equality of Proposition [T] the determination 
problem of 7?,i(r, DjEyi,) can be converted into the deter- 
mination problem of 7?.L(rj4^^, D + tT[B]\ S^iyi)- Using 



Theorem |5] we derive a matching condition for 7?,]J"^ {TA ^ , 
D + tr[i?]|Sxiyi.) to coincide with 7^^°"*^(^i-^ D + 
tr[_B]|I]j(-i,yt). For simplicity of our analysis we use the 
second simplified matching condition ( fTOl i in Theorem |5] Note 
that 

-1 



(ri-i)-i(i]-i+s-i)(ri 



rA-i(i] 



1 



-1 \-lt 



^NL 



y{TA-')^B. 



(28) 



By (l28T l, the second matching condition in Theorem|5] the third 
equality of Proposition [T] and Property |7] part c), we establish 
the following. 

Theorem 12: Let /^*ii,i be the minimum eigenvalue of 



B==r(I] 



ATi 



s^ii]-,,j^s^t j r. 



If we have 
0<D<{L 
then 



i)m,* 



tr [r(E 



AT-L 



s^iE^j^s^t) rj 



n 



'}n). 



L (r,z?|Sy. 
7^L(^,I?|Ey^) 



7lf^(r,i?|Eyi 



-L 
J (out 
-L 



7^^"^^,z?|EyI 



An important feature of the multiterminal rate distortion 
problem is that the rate distortion region 7?.L(r, DjEyi,) 
remains the same for any choice of covariance matrix E^i. 
and diagonal covariance matrix Ejy^ satisfying Eyt = E^t + 
E^L. Using this feature and Theorem [T2l we find a good pair 
(Exi , Ejyj^) to provide an explicit strong sufficient condition 
for 7^^'"'(^, D\Y.yl) and 7^^°"''(^, DjEyi) to match. 

In the following argument we consider the case where F is 
the following positive definite diagonal matrix: 



F = 



71 



72 







0\ 



1l) 



ji G [l,+oo). 



(29) 



A 



Set 7-^ = (71,72,- • • ,7l) e [l,+oo)'^. We call 7-^ the weight 
vector. Since F is specified by the weight vector 7^, we 
write 7?.l(F, Z?|EyL) as 7?.l(7^, i'lEyt). Similar notations 
are adopted for other regions. 

We choose T,j^l so that E^rt = dT"^. Set E^^ = TEj^^F 
and Eyt = FEytF. Then, we have 



B 

E^i. 



SIl 

Eyi, — SIl^ 



^x^' 



A 



m < V2 < ■ ■ ■ < vl 



A 



Let rjn 

of L eigenvalues of Ey 

Vl = Vmax be the ordered list of L eigenvalues of Eyi 



and let ry,„i„ == fji < 772 < 



(30) 

be the ordered list 

< 

Set 



7m ax 

have 



A 



maxi</<L7i- Since r/min-^i ^ Eyt ^ ?/maxi^L, 



we 



riminlh :< ??minr^ ^ Eyt ^ T^maxr^ < 7max'7max-fL , 



from which we obtain 



< Vrain < ?7max < 7max'7n 



(31) 
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We choose 6 so that < 5 < r/min- Then, by ( [30] l, we have 

6^ 



* c 

Mmin ^ " 



'/n 



tT[B] = tr 

From ([32]), we have 



(J/l + J^E-i 



"x^ 



^^+E^ 



Z=l 



fjt- 5 



then we have ( [34] i for any weight vector 7^ e [l,oo)^. If 
7max = 1 and 



> (32) 



Q<D< 



1 



^max^min 



(Vi + Vi- 1)^ '/max - ??min ' 



then we have 



(i + lK„n-tr[5]=5+(|^±ll5-^^ 



1=1 " 



= (5 



L,5J y.' J^ 



??max - <5 f^^Vl- ^ 



>5 + i__^: ._(L-1)- 



^'?n 



^?inax '^ 
^max 



^min - <5 



^ma,yi ^ 



-(L- l)77min 



??min - S 



(33) 



By an elementary computation we can show that the right 
member of ( |33] | takes the maximum value 



L- /L^T) 



2 ^niax^min 

^niax^min 



(VL + Vi - 1)2 ^max - ??n 



at 



(VL - Vi - l)?7max?7n 



\/i^??max - Vi - l?7min 

Furthermore, taking dSTT i into account, we obtain 

^max^min r~ — 1 — 1 1 ~1 ^ r —1 —2 —1 1~^ 



^max ^min 



L /inin /max J — L /min /max /max J 

7/max7?min 



'/max ')'niax'7n 



Hence if 



o<i:>< 



^max^n 



(\/r + \/i - 1)2 77max " 7max?7i 



-2 ' 

mm 



then the matching condition holds. Summarizing the above 
argument, we obtain the following corollary from Theorem 

m 

Corollary 4: Let 7^ e [l,+oo)-^ be a weight vector and 
let 7max= maxi<;<L 7i- If 



n^i''\D\j:Y-)= Tip {d\j:y-) 
= nL{D\Y.YL) = n''°'''\D\Y.YL). 

Fix 7^ e [l,+oo)^ arbitrarily. Consider the region 7?.i,( 
7-^lSyi,) and the minimum distortion 1)^(7^, i?^|I]yL) in- 
duced by TZl{'^^, D\Y,yl). Those are formally defined by 

7eL(7^|Sy^) = {(i?^,^) : i?^ e 7eL(7^,I?|I]yL)} , 
i^L(7^,i?^|SyL) = inf {i? : (i?^,Z?) G 7eL(7^|I]yL)} . 

Similarly, we define 

7^f' (7^|i]y.) 

= {(i?^Z?):i?^e7^f)(7^I?|Sy.)}, 
7e^*)(7^|Ey.) 

^ {(i?^Z?) : i?^ e 4°"*)(7M?|Sy.)} , 
4")(7^i?^|Sy.) 

^inf{i?:(i?^i?)e7^f)(7^|Sy.)}, 

D«(7^i?^|Sy.) 
^inf{i?:(i?^i?)e7^i™*)(7^|Ey.)}. 

From Theorem [TTI and Corollary 21 we obtain the following 
corollary. 

Corollary 5: For any i?^ > and any 7^ e [1, +00)^, we 
have 

i?i")(7^,i?^|Ey.) > i5i(7^,i?^|Sy.) 

>i?«(7^i?^|Sy.)■ 

For each 7-^ e [1, +00)^, if we have 

0<i?i"'(7^i?^|Sy.)< ^ 



(x/i+\/r^T)2 



then 



o<i:>< 



then we have 



1 



'HmsixVn 



(y/Z -\- \/L~ 1)2 ?7max - 7max??i 



-2 ' 

'-I 
mm 



7^i"^' (7^ T^lSy. ) = 7^1;") (7^ i^lSy. ) 

= 7^L(7^,i?|Sy.)=7^i°""(7^,i?|Sy.)■ (34) 

In particular, if 



0<L»< 



1 



(Vi+\/r^T) 



Vn 



= i?«(7^i?^|Sy.). 

We apply Corollary |5] to the derivation of matching condi- 
tion in the case of vector distortion criterion. We consider the 
region TZl{^y^) and the distortion rate region 'Dl{R^\Y,y'^) 
induced by TZ{D^\Y,y^)- Those two regions are formally 
defined by 

TZl{^y^) = {(i?^,/?^) : R"- e 7^Lp^|I]yL)} , 
I?L(i?^|I]y^) = {D^ : iR^,D^) e 7^L(Ey^)} . 
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Similarly, we define 

p['"^(ii'^|Ey^) ^{d^ : (i?^,i?^) e7^^'"'(EyO}• 
Although the distortion rate region is merely an alterna- 
tive characterization of the rate distortion region, the for- 
mer is more convenient than the latter for our analysis 
of matching condition. We examine a part of the bound- 
ary of 2?('"^(i?^|I]yi) which coincides with the bound- 
ary of I?(i?-^|EyL). By definition of Dl(7-^, i^^jSyi) and 
D^"^(7-^,i?^|Syi,), we have 



DL(7^,i?^|I]y.)= min Vt-A, (35) 



1=1 



4"'(7^i?^|Sy.) 



Consider the following two hyperplanes: 



Et'^'- (36) 



A 



nL(7^) = <^ D 



iL . Y^ 2 



1=1 



-,fDi^DLir,R'^\^Yr^) 



It can easily be verified that the region 1?^ (i?^ | Eyi, ) is 
a closed convex set. Then by ( |35] |. 111,(7^) becomes a 
supporting hyperplane of I'L(i?^|EyL) and every D^ E 
nL(7-^)n 2?L(i?^|EyL) is on the boundary of Vl{R^\^y^)- 
On the other hand, by its definition the region Vl'^' (R^ \ Eyi, ) 
is also a closed convex set. Then by (l36T l, 11)^" (7^) be- 
comes a supporting hyperplane of I?]]' (i?^|Eyt) and every 
L>^ en*^"^(7^)n v1"\r^\Y.yl) is on the boundary of 
p['"'(i?^|Eyi,). Set 

Cl - 



Tl{Cl) = {7^ e [l,+oo)^ : 4")(7^i?^|Ey.) < a}. 

Then by Corollary |5] for any 7^ e 71 (Cl), we have 
n^")(7^) = nL(7'^), which together with vf'\R^ JE^l) 
C Vl{R^ |Eyi,) implies that every D^ e n^"^(7^)n 
I?['"^(i?,^|EyL) must belong to 0^(7^)0 I?L(i?^|Eyi,). 
Hence this D^ must be on the boundary of Dl^R^ |Syi). 
It can easily be verified that an existence of 11}^" (7^) satis- 
fying 7^ e 7l(Cl) is equivalent to n^"^(7^) njZ)^ > 0} 
C2?(+)(a),where 



Summarizing the above argument, we establish the following. 

Theorem 13: The distortion rate region I?i(i?^|Eyi) and 
its inner bound 2?]J" (_R^|Eyi,) share their boundaries at 




5+2i6 



Fig. 3. V^^'"-\r^\T,y3), n^"'(73) n {D^ > 0}, and vi+^Cs) in the 



case of L = 3. In this figure we are in a position so that we can view the 



supporting hyperplane Ilg '{7'') as a horizontal line. 



2)2(Cl) n 2?f^(i?^ I Ey.), Where 

2?2(a)= U <(^') 

7-^eri,(a) 

U <'(^')- 

n<:''(7^)n{i5^>o}ci5(,+'(a) 

When L = 3, we show V^^"^\r^\J:y3), ^^^"^'(Cs)- and 
4"^ (73) n{D3 > 0} in Fig. [3] 



D. Sum Rate Characterization for the Cyclic Shift Invariant 
Source 

In this subsection we further examine an explicit charac- 
terization of Rsum,L{ D\J^Y') when the source has a certain 
symmetrical property. Let 

1 2 ■■■ I ■■■ L 

r(l)r(2)...r(0---r(i)^ 

be a cyclic shift on A^, that is, 

r(l) = 2,r(2) = 3, • • • , t(L - 1) = L,t{L) = 1. 

Let pxj,^ {xal ) = PXiX2---Xl {xi,X2,- ■ ■ ,xl) ^^ 3. probabil- 
ity density function of X^. The source X^ is said to be cyclic 
shift invariant if we have 

PXa^{Xt{Al)) ^PXiX2-XLix2,X3,--- ,Xl,Xi) 

^PXiX2-Xl{xi-,X2,- ■■ ,Xl^1,Xl) 

for any (xi, 2:2, ■ ■ ■ ,xl) G X^. In the following argument we 
assume that X^ satisfies the cyclic shift invariant property. 
We further assume that iV;, Z e A^ are i.i.d. Gaussian random 
variables with mean and variance e. Then, the observation 
Y^ = X^ + N^ also satisfies the cyclic shift invariant 
property. We assume that the covariance matrix E^i, of N^ 
is given by e/^. Then A and B are given by 

A = (eE^i +lLy\,B = e {II + eE^i) . 

Fix r > 0, let Ni{r), I E A^ be L i.i.d. Gaussian random vari- 
ables with mean and variance e/(l — e^^'). The covariance 
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matrix E^l/^j for the random vector N^{r) is given by 



1-c- 



-It. 



Let 1^1,1 G Al be L eigenvalues of the matrix Eyi and let 
/?; = f3i{r),l E Al be L eigenvalues of the matrix 



Proof of Property [8]part a) is easy. We omit the detail. Proof 
of Property [8] part b) will be given in Section V. Set 



■log 



J^i 



B\c 



2Lr* 



'A S 



1-C 



-2r 



It. a. 



■"A-i 



Using the eigenvalues of Syt, Pi{r),l G Al can be written 
as 



?(i) 



1=1 
X:ilog{^[e--l]+l} 

l{D,r). 



1=1 



(3i{r) = i 



1 1 - - I e- 

M/ V M; . 



Let ^ be a nonnegative number that satisfies 

L 



r>r*(_D+tr[i5J) 

Then we have the following. 

Theorem 14: Assume that the source X^ and its noisy 
version Y^ = X^ + X^ are cyclic shift invariant. Then, we 
have 



EiK-/5r'r+/3r'} = ^+tr[i?]. 



?(i) 



(u) 



;=i 



Define 



'^(Ar) = n{K-/^r']^+/^r'} 



;=i 



The function ^(Z), r) has an expression of the so-called water 
filling solution to the following optimization problem: 



Proof of this theorem will be stated in Section V. We next 
examine a necessary and sufficient condition for B^^l^-^ ^ [D 

jEyi.) to coincide with i?su„, i( -D|Syt). It is obvious that 
this condition is equivalent to the condition that the function 
J_{D,r), r > r* = r*{D + tr[i?]), attains the minimum at 
r = r* . Set 

A . A 



Mn 



mm ni,n,nax 

KKL 



max 111. 

KKL 



Cj{D,r) 



max TT S.I 

6A>i,ieAi., -'■-'- 



(37) 



Set 



l{D,r) = \\og 



T.f=lil<D+tr[B] 



c^^''\>:yl+b\ 



1=1 



Let /q G Al be the largest integer such that /i„iax = 
let li ~ li{r) E Al be the largest integer such that 

/3;j(r) = max Pi{r). 

KKL 



Mio and 



CjiD,r) 



7r(r) — tr 



A-' i E^i 



1-c- 



'A' 



The following is a basic lemma to derive our necessary 
and sufficient matching condition on R^^^^_^ j^{D\T,y'-) = 

Lemma 6: The function J_{D,r) , r G [?'*(£' + tr[i?]),oo) 
attains the minimum at r = r* if and only if 



By definition we have 



1 

7r(r) = V -77^ 



(38) 



Since 7r(r) is a monotone decreasing function of r, there 
exists a unique r such that 7r(r) = D + tr[i3], we denote 
it by r*{D+ It[B]). We can show that Lu{D,r) satisfies the 
following property. 
Property 8: 

a) For D>0, 

{r,r,---,r)e Bl{A'^ , D + tT[B]) 

L 

^Tr{r) < D + tT[B] -^ r > r* [D + ii[B]) , 



2 \dr- 

L c-'r' 


{D,t) 
e^'-* - 1 + 






) 


e^-* - 


1+ ' 1 

Mil 


1=1 




e2 


'■* - 1 + ^ 


2 







>0. 



(39) 



(i(i^,r*) = |ir 



1-e 



-2r* 



"X^ 



-/l 



b) The function d)(D, ?■) is a convex function of r e [r* {D- 

tr[B]),oo). 



Proof of Lemma |6] will be given in Section V. Note that for 
any / e A^, we have 

>4---)- (40) 

Vmzo Mil/ 

From ( [39l ) in Lemma |6] and ( |40l i. we can see that ^0 = ^1 is a 



e^'-* - 


-1 + ^" 


V Mil/ 


c^'-*- 


-1+ '^ 
Mil 


e^"^*- 


e 

-1 + 

M/o 


V Mil/ 


e^-' 


e 

-1 + — 



(1) 



.(u) 



sufficient matching condition for -Rgm^ ^(D |Eyi ) = ^sum l( 
DjEyt). Let /i be the second largest eigenvalue of Syt and 



_Lp4r* 

£e2'-*|3,(:r*) 
ee2'-*|37('-*) 



yb = a(e^''*-a) 




Fig. 4. The graph of b = a{c — a) 



let I e Ai be the largest integer such that fj, — fij. From the 
graph of 6 = a(e^'" — a) shown in Fig. H) we can see that 



1 

2 

or equivalent to 



< -C 



2r* 



1 > 



1-e 



M /^n 



(41) 



is a necessary and sufficient condition for Iq = /i. Hence dTTT i 
is a sufficient matching condition. Next, we derive another 
simple matching condition. Note that 



> e 



2r* 



,2r* 



e 



1 



Mil 



-,2r* 



1 



e 

Wi 



Mmax ^ 



,2r* 



3 2r* 
4 



e--l-Ml-^ 

'J \ Mniax 



Hence, if we have 

„2r* 



i>5li 



4e 



(42) 



then the condition ( [39] l holds. For e G (0,/i,„in), define 



s(e) = — log s 1 + min ■ 



Then the condition (HTt or (l42l i is equivalent to r* > s(e). 
Furthermore, this condition is equivalent to < Z3 < Dthi^), 
where 



1 -e 


n 

1 


1 




- 


V/^ ^max 


1 


1- 


4e 


n 


3 


A'' max _ 






^.w^E^-MBi^E;::! 



M/e 



32.(e) -1] +£- 



Summarizing the above argument we have the following. 

Theorem 15: We suppose that Y^ is cyclic shift invariant. 
Fix e S (0,/i,nin) arbitrary. If < -D < -Dth(e), then we have 



?(i) 



(u) 



^sum.LplSy-) = Rsnn.AD\^Y-) = i?^Z,L (^l^y- ) 



Furthermore, the curve R = Rsum.L{D\^Y^] 
ing parametric form: 



has the follow- 



1=1 

L 



^-E 



1 



L 



for r G [s(e),oo). 



M/e 



r^AM 



^ M.(e- - 



(43) 



Since I?th(e) is a monotone increasing function of e, to 
choose e arbitrary close to /^min is a choice yielding the 
best matching condition. Note here that we can not choose 
e = /imin because 7r(r) becomes infinity in this case. Letting 
e arbitrary close to ^min and considering the continuities of 
Z)th(e) and the functions in the right hand side of ( l43b with 
respect to e, we have the following. 

Theorem 16: We suppose that Y^ is cyclic shift invariant. 
If < Z? < -DthlMmin), then we have 



"'sum.L 



(DjEyL) = i?sum,L(£'|Syl,) = R. 



(u) 



w,l(^|Sy-). 

Furthermore, the curve R = i?sum,L(£'|Syi,) has the follow- 
ing parametric form; 



R 



D 



^ 1 

^ 2 

i=\ 

L 

E- 



log 



Mi r 
/^//-^min 



-,2r 



1 



1 



^ ^/(e2''-l) + M.i 



for r e [s(^niin),oo). 



Let 1^ = (1,1,--,1) be a L dimensional vector whose 
L components are all 1. We consider the characterization 
of Rsum.hiD ■ l^jSyi,). From Theorem [T6l we obtain the 
following corollary. 

Corollary 6: Suppose that Y^ is cyclic shift invariant. If 
Q < D < -r^th(Mmin)- then we have 



R. 



(1) 

sum.L 



(i^-l^lSy.) 

= RsumxiD ■ 1 |Syi) 

Furthermore, the curve R = 
following parametric form; 



-Kl.LiD-l'^l^Y^). 



Rsum.AD ■ l^lSyi) has the 



1=1 ^ Pmin 



D 



1^ 



M;Mn 



L^Mi(c2'--l) 



Mil 



for r e [s(^,nin),oo). 



Here we consider the case where 'Sy'- has at most two 
eigenvalues. In this case we have ft = ^min- Then we have 
s(Mmin) = and Dth{0) = tr[Syi]. This implies that R = 
-Rsumx(-D-l^lSyi) is determined for alio < D < itr[S]yi]. 
Wagner et al. ifTTl determined R = Rsuni,L{D ■ l^jSyt) in a 
special case where EyL satisfies [SyLJ/; = a^ for I G Al and 
[Syt]/;' = ca^, < c < 1 for I ^ I' G A^. In this special case 
Syt has two distinct eigenvaules. Hence our result includes 
their result as a special case. 
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Yang and Xiong ||25l determined Rsum,L{D ■ l^|Ey-L) in 
the case where Eyt has two distinct eigenvalues. Wang et al. 
lfT2l determined -Rsum,L (-D • 1^ | Eyt ) for another case of Syi . 
The class of information sources satisfying the cyclic shift 
invariant property is different from the class of information 
sources investigated by Yang and Xiong i25l and Wang et al. 
IIT2I although we have some overlap between them. 

V. Proofs of the Results 

A. Derivation of the Outer Bounds 

In this subsection we prove the results on outer bounds of 
the rate distortion region. We first state two important lemmas 
which are mathematical cores of the converse coding theorem. 

For / G Ai, set 



Proof: For each / e A^ — S, we choose Wi so that it takes a 
constant value. In this case we have rj" -^ — for I <E A^ — S. 
Then by Lemma [HJ for any k € Aj^, we have 



>-log<^(2^e) 



We choose an orthogonal matrix Q G Ok so that 

becomes the following diagonal matrix: 

/ 



.(46) 



kk 



n 



A 



(44) 



y-l , t 



ak'j^^uAVq 



-JVs(r<,"')- 



Al 







0\ 



Aa'/ 



(47) 



For Q e Ok, set Z^ = QX^. For 



-A' 



^Kf 



X-=(X-(l),X«(2),...,X«(n)) 



we set 



z^ = Qx"" = (gx^^(i), gx^(2), • • • , gx^'(n)). 



Then we have the following chain of inequalities: 

/(X^; Ws) = KX"") - h{X''\Ws) 

K 

^^ h{X'') - h{Z^\Ws) < h{X'') - J2 h{Zk\Zf^,^Ws) 
(b) n 



k=l 



- A 

Furthermore, for X 

set 



(X^(l), A:^(2), ■■■,X^{n)), we 



< -log[(27rc)^|S];,^|] 

A 

n 



z^ = Qx^ = (gx^^(i), gx^(2), • • • , gx^(n)). 

We have the following two lemmas. 

Lemma 7: For any k € Aa and any Q G Ok, we have 

h{Zu\ Zf.^W"^) < h{Z, - Zu I Zf^] - Zj]) 



■Eo^"^ 



fc=i 



1 

27re 



Q\ ^x^ 






kk 



(c) n 



A 



= :Tlog|ExA'| + 2^-logAi 



fe=i 
n 



-log|Ex^l+ 2 log 



■^x^' 



ME" 



Ns(.tI 



A 



< - log <^ (2tc) 






loa 



/ + ExA-'AE~i ,„, A 



fefc 



where /i(-) stands for the differential entropy. 

Lemma 8: For any k G A a and any Q G Oa, we have 



h{Zk\z;i]W^) 



> - log <^ (27re) 



Sa-+*^s;' ,.<").^rQ 



^AlC'-I"/) 



fcfc 



Proofs of Lemmas [T] and [8] will be stated in Appendixes A 
and B, respectively. The following lemma immediately follows 
from Lemmas [7] and [8] 

Lemma 9: For any Sj^k^l and for any {fi , V2 i ' ' ' ' 

f^ , i/i^")), we have 



Step (a) follows from the rotation invariant property of the 
(conditional) differential entropy. Step (b) follows from ( l46b . 
Step (c) follows from ^}. ■ 

We first prove the inclusion TZL{^d\ ^x'^V^) ^ 7?.)°" (E^ 
I'Exkyl) stated in Theorem |3] Using Lemmas 171 l8l [TOl and a 
standard argument on the proof of converse coding theorems, 
we can prove the above inclusion. 

Proof of TZL{^d\^x'<Y^) ^ 7^i°"''(Ed|ExK-yL).• We first 
observe that 



Ws-^Ys-^ X'"- -> Ys^ -^ Ws-^ 



(48) 



hold for any subset 5 of A^. Assume (i?i,i?2, • • • ,Rl) G 
7^i,(Sd|Ej5fKyi.). Then, there exists a sequence {{ip{'' ,ip2 , 
•••,y3^"\V^"^},T=i such that 



n^x'^-x' 



^ ^XK 



ME" 



A^A^ (>-:"') 



From Lemma [8] we obtain the following lemma. 

Lemma 10: For any 5* C A^, we have 



I{X''-Ws)< -log 



1 + Y.xK^AY.-^ ,„, A 



We set 



(45) 



n 



A 



limsupi?^"' <Ri,leAL, 

n— >oo 

limsup-E^;,-_^A- ^ Ed. 



limsupr^"^ = limsup -I{Yi;Ws\X^). 



(49) 



(50) 
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For any subset S C A^, we have the following chain of 
inequalities: 

^ni?|") > ^logM; >Y,H{Wi) > H{Ws\Ws^) 
les les les 

= I{X'';Ws\Ws^) + H{Ws\Ws^X'') 



By letting n ^ oo in (|55] | and (|56] | and taking i49[ into 
account, we have for any S C A^, 

Y,Ri>Jsi\^d\,rs\rs^), (57) 



and 



les 



E_yK + AJ^^L(^^.L-jA >z S^ . 



(58) 



ies 



(b) 



(c) 



/(X^^; W^slW'sO + Y. HiWilX"") 



l£S 



I{X'';Ws\Ws^)+nY, 



I ' 



(51) 



From (|57l) and (|5l, 7^L(Sd|SJf/fy^) C 7^^™'^(Sd|Sx^Y^) 
is concluded. ■ 

Proof of Theorem |?} We choose an orthogonal matrix Qg 
Ox so that 



;gs 



where steps (a),(b) and (c) follow from (|48] |. We estimate a 
lower bound of I{X^] Ws\Ws^)- Observe that 



Qr-i (e-4 



-ME-i(^,^^l*r-»' 



/(X«;W^s|W^sO=^(^ ^w^"^)-^!^ ;W^sO- (52) 

Since an upper bound of I{Xs<^; Ws^) is derived by Lemma 
[TOl it suffices to estimate a lower bound of I{X^; W^). We 
have the following chain of inequalities: 



ai 



a2 











a_ftr 



Then we have 



I{X^;W^) = h{X^) - h{X^\W^) 



__ff 



>|log[(2^e)^|S;,.-|]-flog[(27re) 



K, 



= — loe 
2 ^ 






K 



1 V 







1 V 
n-^X'^-X'^ 



(53) 



Combining (|52] |. ( |53] ), and Lemma [TOl we have 



J(X^^;t¥5|M^sO+"E 



ies 



n 



2 ** 



J") 



(n) 







For Ed e y4^(r-^), set 



■"K 



(59) 



Ed = grE/r*g, ^u - 



n.ese''^' |5^ 



x^ 



^+^-^-'^^i(.<"')^ 




1 V 

n'^X^'-X'^' 



fcfc 



Since 



n 



^2r) 



les' 



^--+'^^i(.-)^ 




n^X'<-X'' 



FE/r h r(E-i +*AE-i(^^)^)-"r, 

Eg, and tr[rEd'r] < D, we have 
£,k > ctfe \ for k e Ak, 



K 



J2^k= tr Ed = tr [rEd*r] < D. 



(60) 



Note here that /(X ; VFs|M^5c)+n^,-g^ r^ is nonnegative. 
Hence, we have 



I{X'';Ws\Ws^)+nY^rl 



(n) 



fc=l 

Furthermore, by Hadamard's inequality we have 

K K 

|Sdl = |r|-2|Edl < |r|-2 n P'^]^^ = l^r' n ^^- (61) 



fc=l 



fc=l 



> nJc 



n^X^^-X^^I '^S'l'^ 



(") I ^(") 



'^ -* Combining ( |60l l and (jSTJ, we obtain 



Combining ( fSTl i and ( l54l i. we obtain 



Y.^'r'>is{ 



l£S 






„(») 



J») 



for 5 C A^. On the other hand, by Lemma |9] we have 



(55) 



(56) 



e{r,D,r^)^ max |Ed| 

tr[rEd'r]<n 

< |r|-2 max TTffe =a;(r, D,r^ 
The equality holds when Ed is a diagonal matrix. 
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Proof of Theorem U4\ Assume that (i?i, i?2, ■ ■ ■ , Rl) G 
7?.i,(£'|Syi,). Then, there exists a sequence {{ip]'^' ,ip2 , 
•••,V'i"\'/'^"^}5f=i such that 



limsupi?!"^ <Ri,leAL 



1 



lim sup — S 

n— >-oo 'T' 

for some S^j 






:< Erf, trprf] < D 



} (62) 



Step (a) follows from (|64]i. Step (b) follows from ^^. Step (c) 
follows from the definition of S^. From Y\,^, we construct 
an estimation X\^ of Xa^ by X^^ = ^^At • Then for 
j = 0, 1, • • • , i — 1, we have the following: 



(a) ^-1 



^-1 



-^■(Al,) ~.. (A ,,)(<" A,,) 



For each j ^ 0, l,---,i - 1, we use (^^"|jp (p^"|2)' ' ' ' ' 
V^^'jJ^P for the encoding of (^1,^2, • • • , Yl). For Z e A^ 
and for j = 0, 1, • • • , L — 1, set 



— n X 



(At)^VJ(Ai)' 



^'^^-A.-^.,A.j'^ + ^-^J-^ 



(66) 



In particular. 



Steps (a) and (c) follow from the cyclic shift invariant property 
of X\j^ and Xa^ ^ respectively. Step (b) follows from Lemma 
|9] From ( |66] |, we have 

L-l 






-I{Yi;Wi\X,), forleAi. 



'-Ya 



Furthermore, set 






(r)7,r'^:^,---,r)'X), for j = 0, 1, • • • , L - 1, 



>- 



(„) ^ j_ y- („) 
1=1 
By the cyclic shift invariant property of Xal ^nd Y/^j^, we 
have for j = 0, 1, • • • , L — 1, 

L _, L 



(a) 






-E 

1 i-i 



E-' 



I, N 



K. 



T3(A_c)'^'T'n(A_L) 



i=o 



^ 



.(«) 



(63) 



■i'Ar-'i'- 



Va,-!' 



'^ + SxAjyA, 



M + E 



t ' TJ(Ai) 



Xa^I^A^ 
1 



Yk,-Y 



3=0 



L ' xJ(A_l) 



+ B\'A 



(67) 



(=1 (=1 

For j = 0,1,- ■ ■ ,L — 1 and for ^ G A^, set 



Step (a) follows form that {AY} A + E^^ \Yk )^^ is convex 
with respect to E. On the other hand, we have 

L-l 






J,i 



^E 



i=o 



^-1 1 ^-1 



■"Xa 



I, w. 



= E 



Xj\^ 



1 L 1 -2r<"' \ 



L 



1=1 



1 \-i „.(") 



By the cyclic shift invariant property of I^Al , we have 

^{Yi-Yj^uYv-Y,.v) 
= ^{Yr(i)-Y^,i^Yr^^i,)-Y,j,) (64) 

for {I, V) e Al and for j = 0, 1, • • • , L - 1. For E^ = [^h/]. 



< E 



1 _ c-H Eti '■; 



-fAr 



= E 



1-c- 



Xa, 



/f. 



(68) 



set 



L-l 



T-'(Ed) = K3(;)TJ(i')]' ^"i =" 7 E ■^■'(^'i)- 



Step (a) follows from that 1 — c ^^ is a concave function of 
a. Combining ( |67] | and ( |68] |, we obtain 



j=o 



Then, we have 



1 ^"^ 
limsup-^iE 

L-l 

= limsup-^iEy ^ 

(b) 1 ^ . (c) _ 

J=0 



^ 



^A^-^W( 



•^ItE^^ 



^A, Y 



B M 



Al,) 



(Af,) -■ TJ(Ai) 



from which we obtain 

L-l 



L '■ tHK^) 






Ya,-Y 



j=0 



(65) 



y 



'M^xi 



l-e- 



B 



Il>A 



(69) 
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Next we derive a lower bound of the sum rate part. For 
each j = 0, 1, • • • ,i — 1, we have the following chain of 
inequalities: 

E"^l"^> E^og^^'^ E^(^^-^') 

ieAi leJ^L le^L 

(a) 



Set 






^d — Q^d Q, Bd — QB Q, £_i 
From ( |73] | and ( f74b . we have 

6>/3r'W,'eAi, 



Srf + B 



leAL 



E^'- 



tr 



Sd + B 



= tr[Ed + S] < D + tiiB]. 



(75) 



/=i 



From drsl l, we have 



*='/(Xa,;W^,.(a, 



> ^log 










S^A. 








^^x 


At~^, 


^■(At) 








log 



nLr'-''^ 



^^>-A.*^ + S^Aj^A. 



1=1 ^'^ -^ ;=i 

^r> ?'*(£> + tr[B]). (76) 

Furthermore, by Hadamard's inequality we have 

L L 

iSd + B\ = \td + B\<l[[td + B]u = n ^'- (77) 



A is 



Sy., + B 



is. 



" ^Aj,-y^,(^^, 



B 






nLr 



(") 



i=i 



i=i 



Combining dTSl l and ( ITTI i. we obtain 



(70) 



\^d + B\< max W^i^[b{D,r). 

j:tii'<D+tr[B] '-1 

Hence, from (iTZt . (|76] |, and dTSl l, we have 



(78) 



Step (a) follows from (|48]l. Step (b) follows from (|65]l- Step 
(c) follows from (|53] |. From ( T/Ol i. we have 



L-l 



1 

ERi > min - log 

r>r*{D+ti[B]) 2 



c:;(i^,r) 



E^I-'-tEE 



R 



(n) 



ieAi 



> - > - log 



j=o letvL 



min JiD,r) == i?sum.L(-D|SyL), 

r>r* {_D+tr[S]) 






Lr 



(n) 



completing the proof. 



B. Derivation of the Inner Bound 



(a) 1 

> -log 



^Y.,+B 




lI^u^y^^-y^, 

3=0 


+ B 

Al) 



Lr("). (71) 



In this subsection we prove 7?.)]' (S^ jSY^yi-) ^ TZLi^d 
|SxAyt) stated in Theorem[3] 

Proo/ o/ 7^^"^(Erf|SxifyI.) C 7^i(I]d|Sx^F^)■• Since 
'R-l^ { EdlSx^'y^) ^ 7^L(Ed|5]xifYi.) is proved by Theorem 



Step (a) follows from that — log |E + i?| is convex with respect 
to E. Letting 71 — > oo in ( |69] l and (ItTI i and taking ( |65] | into 
account, we have 



[U it suffices to show 7^^'"^(Ed|Ex/cyI,) == 7e^'"^(Ed|Sx/cyi,) 
to prove TZ)^ iTjd\^x^Y^) ^ ''^L(Ed|Ex/fyi-). We assume 
that i?^ € TZj^ (EcflExicyi-). Then, there exists nonnegative 
vector r^ such that 



V^ 1 S^-A, + S 

leAL 



( 



E^R^ + AT,j^j:^^^ls^A\ :< Ed 



Ed + B 



ir, 



(72) and 



Erf + B^ 



'^^^xL 



1-c 



-2r 



E ^' ^ ^sl^-sksO for any S C A^. 



(79) 



/r. U 



,(73) 



ies 



tT^ + B]= tr[Ed] + tr[S] < £> + tr[S]. (74) 

Now we choose an orthogonal matrix Q E Ol so that 



Q'a\^-' 



X\, 



1 - C-2' 



Pi 







/32 

/3. 



Let Vi,l E Al be L independent zero mean Gaussian random 
variables with variance ay . Define Gaussian random variables 

Ui,l e K^'^y Ui= Xi+ Ni + Vi. By definition it is obvious 

IJL _^ y-L -^ X^ \ 

Us^Ys^X^ ^ Ys^ ^ Us^ \ (80) 

for any S C A^. J 

For given r; > 0, / G A^, choose (jy so that ay = 
aj^ /(c^'' — 1) when ri > 0. When ri = 0, we choose Ui 
so that Ui takes constant value zero. In the above choice the 
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covariance matrix of N^ + V^ becomes E^i(^t). Define the 



linear function tp of U^ by 




^(t/^) = (S^l +'AI]^l(^,^A)-"AE^i(^,)t/^. 


Set X^ = i' {U^) and 


dkk = E 


]\Xk-Xk\\' 


,l<k<K, 


4fc' = E 


'{x,-x,)( 


Xk'-Xk')' 


,l<k^k' <K 



Step (a) follows from the following inequaUty which is equiv- 
alent to 



D- 



1 



,^a.(r^) 



> 



1 



k£AK~S '^^ ' 



Step (b) follows from Lemma |2l Hence, by ( l84b and Lemma 
|2] ^21 is nonnegative if 



dri 



Let lly^K_xK be a covariance matrix with d^k' in its (fc, fc') 
element. By simple computations we can show that 



^2 2r, 



>o, 



^x'^-x^ ^ \^x 
and that for any S C A^, 



— V^X'^ + ^^Af-tfr^!^) — ^' 



(81) 



^s(r-sksO=^(>s;t/s|t/sO- (82) 

From (gOll and (gB, we have IJ^ e ^(Sd)- Thus, from 
7^^'"^(I]d|Sx^-y^) ^ 7t,^"^(I]d|Sx«'Y^) is concluded. 

C. Proofs of the Results on Matching Conditions 

We first observe that the condition 

-1 



completing the proof. ■ 

Proof of Lemma |?} Without loss of generality we may 
assume k = 1. For Te OK{ai,k), the matrix C*{T-'^T,ri) 
has the form: 



C*iT-^T,ri) 



cl,{r-^T,ri) 



'ct,,(r-ir) 



C2*2(r-^T) 



tr 



r I s^K + As^i,^^i^^ 



T 



<£) 



where (:72*2(r"ir) is a (X - 1) x (A' - 1) matrix with 
4k'(^~^T), {k,k') e {Ak - {!})' in its {k,k') ele- 
ment. Since C*(T~^T,ri) ^ Q;*-ja^^(r;)/A', we must have 
C2*2(r-iT) ^ a*^,,{ri)lK-i. Then we have 



is equivalent to 



E 



1 



c*(r-ir,n)^ 



< D. 



(83) 



Proof of Lemma |5} Let A^ = {1, 2, • • • , A'} and let S C 
Kk be a set of integers that satisfies a^^ > ^ in the definition 
of 6'(r,D,r^). Then, e{T,D,r^) is computed as 



cIi(r-iT,n; 



*ctni(r-^T) 



'^tm(r-^T) 



a„ 



M)Ik-i 



(85) 



Let A be the minimum eigenvalue of the matrix in the right 
hand side of dSSl l. Then, by dSST l. we have A > ajjjijj(rj) and 
A satisfies the following: 



(A-c^i(r-ir,rO)(A-<_(r,)) 



cIm(r-^r)lP = o. 



(86) 



(K-|S|)^ 



U"es 



a," 



k£S 



ak 



Fix / e A^ arbitrarily and set '^i = 2ri— \og9{r,D,r^ 
Computing the partial derivative of ^P; by rj, we obtain 



From ( I86] l, we have 

cti(r-iT,r,) = A- 



clmi^-'TW 



a 



dri 



E 

'jes 



dn 



>E' 

J6S 



> 



jes 



dn 

daj_ 
dn 



J__J£_\s\_J_ 

aj £,_y^J_a2 

kes 

K~\S\ 1 

/ ' ak ■' 
keAK-S 



> "mini'";) 



c(n)-A 

l|ct[i](r-ir)||^ 



> an,in{r^) + 



Cax(n)-amin(n) 



a 



amin(?'-^)' 



completing the proof. ■ 

Next we prove Theorems |5] and |6] For simplicity of notation 
we set 



(b) ,^ f daj 
dn 



jes 






a(r^) 



1 



amin(?'-^) 



.6(r^) 



A 



A 1 



amax(r-^)' 



a 



Then the condition ^ in Lemma [3] is rewritten as 



> 



jes 



dn 






^2 2r, i 



'Ni 



\ai 



12 ^ 

' i=i 



dri 



a(r^) 






< 






(87) 



\o-i\ 



1 



Proof of Theorem |5} For (/, fc) e A^ x Aa', we choose 
T S OkI&i, k). By Lemma |4] we have 



> 



lladP 



E 

3&S 



dn 



.(84) 



rr2 p2'~i 



> 



|a/l 



1 



a{r'^)b*\\c 



Xk 



k[k]l 



a{r^) a{r^) — h* 



(88) 
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It follows from dSTJi, ([88]l, and Lemma|3]that if for any I e Al, 
there exist k E Kk and T S Ok {o-i , k) such that 



Proof of Theorem |6} The inequality (|6]l in Lemma |3] is 
rewritten as 



«(r^) 






< 



Xk 



1 <r^)b*\K[,]\\' 



i{r^) a{r^) — b* 



[a(r^)-6(r^)]^^^--2-' 



'b{r^) 



<Tie^ 



(89) From ( |94] i. we can see that if we have 



holds for r^ e B{T,D), then e(r,D,r^) satisfies the MD 
condition on Bl{ F, Z?). Since the left hand side of (|89] l is a 
monotone decreasing function of b{r^) and b{r^) > b*. 



[air^)-b{r^) 



b{r 



,L\ 



<T* 



(94) 



(95) 



a{r^) 



a{r^) 



-1 



< 



Xk 



1 «(^'')^1|Cfc[fc]ll' 

i(r^) a(r^) — 6* 



on i3L(r, -D), then 0(F, Z?, r^) satisfies the MD condition on 
Bl{T, D). On the other hand, from dSST l, we obtain 



implies ( |89] l. Observe that ( |90l ) is equivalent to 



(90) 



Under 



«(?- ) <D-{K- l)b{r^). 
we have 



(96) 



«(r-^) 



<=> 



a{r^) 
b* 



Xk 



1 «(^'')^1I4mI 



i(r-^) a(r-^) — b* 



i)x:.-^-«(^^)ii4Mir<o. 



< 1 



(91) 



< 






b{r^) 



D - [K - l)b{r^) 
b{r^) ■ 



Solving (l9l1 l with respect to a{r^), we have 

b*xl + 1 



a{r-) < - 



Xfe + IF 



Hence the following is a sufficient condition for ( |95] ) to hold: 

(97) 



b*Xk \\^k[k]\\ 



— h*\\r* l|2 



[D^mr-)]^-^'^r}:^'^'"Kr*. 



xl 



On the other hand, by (|83] |, we have 

a{r^) < D- [K - l)b{r^) <D - [K - 1)6*. 
Then we have the following. 

U*\2\\.. 

'k[k] I 



b{rL) 
Solving WT\ with respect to D, we obtain 



(92) 



(93) 



D < Kb{r^ 



1 



^b'^[rL) + AT*b{r^) - b{r^) 



(98) 



Since the right hand side of ( |98] | is a monotone increasing 
function of b{r^) and b{r^) > l/a*^^^^ by Lemma |2] the 
condition 



D< 



D <Kb* 






1 + 4a* r* - 1 



V* — /)*llr* l|2 



^D- {K - 1)5* < b* 



l + (6*)2||c* N2 



^ im holds under (|93|). 
^ im holds for r^ e 6(F, D) 
^ (|90| holds for r^ e ^(F,!)) 
^ (HU holds for r^ e ^(F,!)) 



6*||c 



fc[fc] I 



is a sufficient condition for (|95] | to hold. ■ 

Next, we prove Lemma |6l To prove this lemma we prepare 

a lemma shown below. 

Lemma 11: A necessary and sufficient condition for J( 

D, r) to take the maximum at r = r* is 



t-M-: 



> 0. 



A 



Hence, if for any / e Al, there exist k e A^ and T e Ok{ 
ai, k) such that 



Proof: For simplicity of notation we set J{r) = J_{D , r) 
Suppose that 

fdJ{r)\ 



D < 



K 



1 






Ar 



>0. 



(99) 



^.(r-iy)_ IIWr T)| 



then 6'(F,i:),r^) satisfies the MD condition on Bl{ F,!)) 
Thus, by Lemma [T] 



D < 



K 



mm max rr— ; /r-lT'MI2 



TeOK'i&i.k) XfclJ- J J 
is a sufficient matching condition. 



Under ( |99] |. we assume that J(r) does not take the minimum 
at r = r* . Then there exists e > and r > ?■* such that 
iZ(^) < Jlr*) ^ £■ Since J(r) is a convex function of r > r*, 
we have 

J(t7~ + (1 - T)r*) < Tj{r) + (1 - T)J{r*) 
< T{J{r*) - e) + (1 - T)J{r*) = J{r*) - re (100) 

for any r e (0, 1]. From dlOOK we obtain 

J{r* +T{f-r*))-J{r*) 



'(f — r*) 



< 



(101) 
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for any t e (0, 1]. By letting r ^ in dlOll i. we have 
[dJ{r)\ , e 



dr 



< 



<0, 



J_{r) takes the 



which contradicts ( |99] l. Hence under _ 

minimum at r = r*. It is obvious that when ( ^g^ ) < 0, 

J_{r) does not take the minimum at r = r*. ■ 

Proof of Lemma |6} We first derive expression of uj{D, r) 
using Pi = /3i{r), I e A^, in a neighborhood of r = r*. Let 
S{r) = {I : A(r) < Pi.ir)}. By definition, L^\S(r)\ is equal 
to the multiplicity of the Pi^ir). In particular, for r = r*, we 
have 



^ h '""'-, 5.. «^) 



PiAr*) L-\S{ 

■(102) 

Since /3;(7'), / G Al are strictly monotone increasing functions 
of r, there exists small positive number 5 such that for any 

re [r* ,r* -^ 5), we have 

S{r) ^ S{r*), 
1 



77 



< 



1 



Mr) 



for fc^ S'(r*). 



The function CJ{D, r),r E [r* , r* + 5) is computed as 



T^F^rm^ n (3Jr) 



X D + trfSl 




L-\S{r')\ 



In the following we use the simple notations /?/ and S for 
(3i{r*) and S{r*), respectively. Computing the derivative of 
J_{D,r) at r = r* , we obtain 



2 Vdr 



;es 



(a) 1 



32r> 



E' 



L 



e 
e 



L^\S\ 



A D + tr[B]-J2j^Pt 



IGS 



L 



tE-^ 



1_^ 
/3i /3? 



1 Ai 




i e^''* e^'^*- 1 + ^ - ( 1 - — ) e^'-*- 1 + ^ 



1=1 
> 0. 
Step (a) follows from (1102 



,2r- _ 1 + -^ 



VI. Conclusion 

We have considered the distributed source coding of cor- 
related Gaussian sources Yi,l £ Al which are L observa- 
tions of K remote sources Xk,k E A a-. We have studied 
the remote source coding problem where the decoder wish 
to reconstruct AT^^ and have derived explicit outer bounds 

'jZ^°^^\r,D^\^x'<Y^) and 7e^°"*^(r, i?|Ex«Y^) of ULi 
r, D^|EjfKyt) and 7^L(r, -D|Sx^y^)' respectively. Those 
outer bounds are described in a form of positive semi definite 
programming. On the outer bound 7?.}°" (r,D\'Ex'^Y'^)^ we 
have shown that it has a form of the water filling solution. 
Using this form, we have derived two different matching 
conditions for 7?.}°" ( T,D\T,xkyi^) to coincide with TZl{ 

In the case of K = L,A ~ II, we have considered 
the multiterminal source coding problem where the decoder 
wishes to reconstruct Y^ = X^ + N^. Using the strong 
relation between the remote source coding problem and 
the multiterminal source coding problem, we have obtained 



, (out) 



(out) , 



the outer bounds 7^^ ' (T, D^ |Syi, ) and U"-^ ' (T, D] EyL ), 
of 7^L(^,D^|EyI,) and 7^L(^, DlSyi), respectively. Fur- 
thermore, using this relation, we have obtained the match- 
ing condition for 7?.]°" (r,_D|Eyi-) to coincide with TZl{ 

In the remote source coding problem, finding an explicit 
condition for 7^^°"*^(^, D^IExayl) to be tight is left to us 
as a future work. Similarly, in the multiterminal source coding 
problem, finding an explicit condition for 7?.}°" (F, D^jSyt ) 
to be tight is also left to us as a future work. To investigate 
those problems we must examine the solutions to the problems 
of positive semi definite programming describing those two 
outer bounds. Those analysis are rather mathematical problems 
in the field of convex optimization. 



Appendix 
Proof of Property^part b): Since 

1{D, r)=Lr- \ogQ{D, r) + ^ log |Sy. + B\, 

it suffices to prove the concavity of log a) (Z?,r) with respect 
to r > r*. We first observe that \oguj{D, r) has the following 
expression: 



logcj(7:>, r)= max y^log^j 

For each j e {1,2}, let ^l^\l = 1,2,-,L be L positive 
numbers that attain log a)(-D,r '■''). Let ti,t2 be a pair of 
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nonnegative numbers such that <i + ^2 = 1- Then we have Expression of Fk{T.\Q) using the above density functions is 

the following. 

L Ffc(I]|Q)= sup h{Zk\Z^.) 



^(tiioger^+t2iogei 

1=1 

<Ei°gK' + t2ef)- (103) 



1=1 



(a) A / ,^^ roA = ^"^P ~ / ^ZA- (z ) log q^^ |^a- (zfc|Z[fc] )dz 



qzK[z ) log ^ p. az . 
Since s^x^ds ^■'"' '''^ 

_]^ _ ^le c '' The following two properties on Fk{T,\Q) are useful for the 



mr)}- 



fii - e fj,i [e^'" - 1] + e proof of Lemma Ul 

fJ-i^ fM^ Lemma 12: Fk{'£j\Q) is concave with respect to S. 

fj,i — e ^/[e^'" — l] + e Lemma 13: 



{Piir)}-^ is a convex function of r > r* . Then we have -Ffe(S|Q) = -log|(27re) [QE~"Q] J|. 

ti^i^^ + ^2^^^ ^ ^i{/3i(^^^'')}~^ + i2{/3i(?'^^^)}~^ We first prove Lemma [T] using those two lemmas and next 

>{(3^{tlr'^^^+t2r^^^)^\ (104) prove Lemmas [H] and [l3] 

Proof of Lemma^ We have the following chain of inequal- 
for I = 1, 2, • • • , L. Furthermore, we have ities: 



EM''+*^?PO -^lE^r^+^^E^I^' ^^- (105) 



h{Z,\ Zl-W"") < h{Zk - z, I Z^, - Z..0 



;=i i=i i=i 

From (I104l i. (I105l l, and the definition of log w(Z?, r), we have 



1=1 
From (1103b and ( 1106b . we have 

ti\ogCb{D,A^'^) +t2\ogCb{D,A^'^) 



(2) 



t=i 

(a) " / 

t=l 

(b) 



(106) - '"^'^ UE^xA(t)-x^(t) 

\ t=i 



Q 



<logi:^(DM'^+t24'^), ^=^5log (27re) 



Q 



fefe 



completing the proof. ■ step (a) follows from the definition of Ffc(S]|Q). Step (b) 

follows from Lemma [12] Step (c) follows from Lemma [13] 

A. Proof of Lemma^ Proof of Lemma Ul} For given covariance matrices I](") 

In this appendix we prove Lemma]?] To prove this lemma and T,^^K let P^L,^,^^ and P^L|^k be conditional densities 

we need some preparations. For k £ Ak and for Qe Ok, set achieving Ffc(E(°)|(3) and Fk(T,(^^'>\Q), respectively. For < 

a < 1, define a conditional density parameterized with a by 

F.(i]|g)= sup M^.-^.l^[f]-^[f])- (.) ^^ ,0) ^ (1) 



To compute Ffc(S|Q), define two random variables by Let p^^l^^ be a density function of (X^, 1^) defined by 

{p^KivK' -Pxk)- Let S ? be a covariance matrix computed 

yK a yK vK yK A „K yK ' /„-, 

^ -^ -A,z -z -z. fj.Qjjj jjjg density pV^^, . Since 

Note that by definition we have Z^ = QX'^. Let Pxkxk „(") ^ n - n,^n^°) + n-n'^' 



(x^,£^^) be a density function of {X'^,X'^). Let q^a^a- '^-^''' '^ ^"'^'^ ' "'^^'" 



(z^, z^) be a density function of (2'^, Z^) induced by the we have 
)nal matrix Q, that is, 

(Z^A^A-(z^,z^)=p,Q^A,Q^A-Cgz^,*Q5^). ^(l-a)l](")+al](i). (107) 



orthogonal matrix Q, that is, /^^ ,-„-, n-i 
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Let g^^ -^ be a density function of [Z^ ^ Z^) induced by the 
orthogonal matrix Q, that is, 

By definition it is obvious that 

(a) /I N (0) , (1) 

- (i-a)gU +ag^ 



IM^'-^M 



= JlogU2-c) 



ilog^(2^c)|0E^^^*Q 



.-1 t/ 



Z-K' 



(c) 1 
< 



9^x 



l^A' 



ZK- 



-log{(2^c)[0E-"Q];;} 



Then we have 

(l-a)Ffc(I](«)|Q)+aF,.(S(i)|Q) 



JG) 



-(1-a) 






Step (a) follows from the fact that q^L and g^^^ yield the 

same moments of the quadratic form logg-^^ . Step (b) is a 
well known formula on the determinant of matrix. Step (c) 



follows from S^i, ^ S. Thus 



i^K 



[fcl 



g^ z" log ^, dz-*^ 



'Z^^.(-^l) 



^Z-v~[fc); 



(a) 
< 



(b) 



4"i(-^)i°g^ 



41 (-^]) 



dz 



A" 



gg,(z^)logg^f:) ,(z,|z^,)dz 



K 



^[fc]J 



< Ffc((l-a)E(")+aS(i)|Q) 



Step (a) follows from log sum inequality. Step (b) follows 
from the definition of Fk(T,\Q) and (fTOTj i. ■ 

Proof of Lemma \13\ Let 



F,(S|0) < ilog{(27re) [QS^^g] J} 

is concluded. Reverse inequality holds by letting Pxkixi^ ^^ 
Gaussian with covariance matrix S. ■ 



B. Proof of Lemma \8\ 

In this appendix we prove Lemma[8l We write an orthogonal 
matrix Q G Ok as Q = [%/:'], where qkk' stands for the 
{k, k') element of Q. The orthogonal matrix Q transforms 
X^ into Z^= QX^. Set Q = Q^A and let qui be the (fc, /) 
element of Q^A. The following lemma states an important 
property on the distribution of Gaussian random vector Z^ . 
This lemma is a basis of the proof of Lemma [8] 

Lemma 14: For any k £ Ak, we have the following. 



9W(2 ) - 



1 



-in^^isitfz^ 



(2^e)^|E^Kp 



and let 



«Z.|Z-/^'=I^W^ - JG) , ;^ 






where 



9kk 



k'^k 



Vkk' Zk 



1 Y^ qu 

9kk f-[ ctn, 



Yi+Nk, (109) 



9kk 



= [Q^xi-*Q].. + E§^' 



L _2 

9m 



(110) 



1=1 "N, 



be a conditional density function induced by q\^J{-)- We first 



observe that 



q^Kiz )log 



1Zf,\Zf<J^k\Z[k]) 



4!Iz-(^fci#]) 



dz" > 0. (108) 



From dlOSt . we have the following chain of inequalities: 

/^(Zfe|Z[f]) = - /'g^K(z^')logg^,|^,^ (zfc|z[^])dz^ 



i^kk', k' £ Ak — {k} are suitable constants and Nk is a zero 
mean Gaussian random variables with variance ^— . For each 

gkk 

k e Ak, Nk is independent of Zk' , k' € Ak — {k} and Yi,l G 
Al. 

Proof: Without loss of generality we may assume k = I. 
Since Y^ = AX^ + N^, we have 



.K^,(G} 



< I q^.{z-)\ogq'y'{zk\zf,^)dz'' 

[k] 

= - / qz'< (^ ) lor 



Since Z^ = QX^, we have 



Aaj^^ Azj^a' a -|- S^i: 









dz^ 



-J q^.iz^)\ogqf^iz^)dz^ 

+ /g^K(z^)loggg)(z[f,)dz^ 

^-^-^P(^")logC^(.")dz- 

+ I qf^{z^nogq^^{zf,^)dz^ 



The density function pzKyi-iz^ ,y^) of (Z^,y^) is given 
by 

PzKYi'iz^,y^) 






(2^e)^^ iSzK 



^KyL 



where I]„i^j^ has the following form: 



^-1 






1-1 



-1 



-ATI 
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For (fc, k') e A|, and / e Ak, set 



A 






Eqkiqk'i 



/3.. = -[Q*AE-],,^ = -*i 



^w, 



(111) 



Now, we consider the following partition of T,yK 



ZKyL 



y-1 



Q(S-4 + 'AE-i A)*g -Q*AS-i ^ 



511 



ffl2 



S/vi^ *y 



"512 



^JV-L 



G, 



where gn, 512, and G22 are scalar, K + L — 1 dimensional 
column vector, and {K + L — 1) x{K + L — 1) matrix, 
respectively. It is obvious from the above partition of Y^'^l^yL 
that we have 



511 = ^i 



^u 






(112) 



5i2 = * [i^i2 ■ ■ ■ i^ia:/3ii A2 ■ ■ • I3il\ 



^-1 



It is well known that Ti^KyL has the following expression: 






511 



512 



'5l2 



G 



22 



1 


'O12I 


l^yi^ 


Il-i\ 



511 



012 
1 



'012 



G22 — -^ 5l25l2 



Set 



A 



\i„Kt„M 



"■1 = Fil ^[1] y 

Then, we have 



[ z y \2^z'<Y^ 



m5i2 



■,K 



12 



Zl 



1 t 

mi. 



512 



/l-1 



511 



Z[i] y 



512- 

(113) 



['^il 2[i] y J 



511 



[zi\'4]y' 



'O12 



'511 


*512' 




Zl 
■yli 


.512 


G22 



G22 - —512*512 



ni 



7^ 



(114) 



From (Illlb - dllSb . we have 



1 1 

fii = Zl H V" i^ijZj H V" /3i/ 

511 ^^ „,, ^1^ 



J=2 
L 



511 



yz 



i=i 



1 Y^ 1 V^ 91/ 

511 f-f, 511 ;r^ ^^, 



i=i "^Wi 
It can be seen from (1114b and (|115b that the random variable 



A^i defined by 



511 1-^ 511 f-f 



qu 



3=2 



511 ^ ^AT, 



is a zero mean Gaussian random variable with variance ^!- 
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and is independent of Z,^ and Y^. This completes the proof 
of Lemma [14] ■ 

The followings are two variants of the entropy power 
inequality. 

Lemma 15: Let Ui,i = 1,2,3 be n dimensional random 
vectors with densities and let T be a random variable taking 
values in a finite set. We assume that t/3 is independent of 
t/i, U2, and T. Then, we have 

J_Q^h(U2 + U3\UiT) > J_Qih{U2\UiT) , 1 „^h{U^) 
27rc — 27rc 27re 

Lemma 16: Let Ui, i ~ 1,2,3 he n random vectors with 
densities. Let Ti,T2 be random variables taking values in 
finite sets. We assume that those five random variables form 
a Markov chain (Ti,t/i) ^ t/3 ^ {T2,U2) in this order. 
Then, we have 

_J_^h{Ul+U2\U3TiT2) 

27rc 

> J-ciHUilU^Ti) , _i_oiHU2\U3T2) 
— 27rc 27rc 

Proof of Lemma ^ By Lemma [T4l we have 

L 
I -^ — , I ■ 



- Vz.fcfc,Zfe, + — V^l^z + AT,, (116) 



3'^'^ fc'^fc 



where AT^. is a vector of n independent copies of zero mean 

Gaussian random variables with variance -^. For each k E 

gkk 

Ak, Nk is independent of Zk',k' E Ak —{k} and Yi,l E 
Al. Set 

h^-^ = -hiZ,\Z^^,W^). 
n ^ ' 

Furthermore, for I G A^, define 

A 



Qkj 



Si ^ {1,1 + 1,- ■■,L},^, = ^i{Ys,)=y2^Y, 






Applying Lemma [15] to ( |116b . we have 



.,2/i<"' 



> 



1 1 



,gf/i(*i|ZfJj,W^) _^ J_^ 



27re (5fefe) 27re gkk 

On the quantity /i(*i Izj^, , W^) in the right member of (1117b . 
we have the following chain of equalities: 



= /(*i; ZfelZf^], W^^) + /i(*i|X^, W^^) 
= h{Zk\Z^},W'^)-h{Zk\^i,Zf^^,W'^) 

^=^W")-/i(Zfc|vl'i,Z|) + M*i|^'',M^^) 

= 7i/i(")-|log[27re(5fcfe)-i] +;j(*i|X^^M^^).(118) 

Step (a) follows from that Z^ can be obtained from X^ 
by the invertible matrix Q. Step (b) follows from the Markov 
chain 



Z,^(vl'i,Z/^]) 



Y^ -^ W^. 
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From dllSl l. we have 



References 



27re 



27re ^ 27re 



Substituting ( |119t into (1117b . we obtain 

f > f ^ ^ _|_ ■|/i(*i|X'\lV^) I ^ 

27re ~ 27re gfcfc 27re gfet ' 

Solving ( 1120b with respect to ^ , we obtain 



(119) 



(120) 



.2/j(") 



27re 



> 



1 2.h(-i>i\X'^,W'^) 

^'"'^ 27re 



(121) 



Next, we evaluate a lower bound of C"''(*il"'^ '^ ^ Note 
that for / = l,2,--,i — Iwe have the following Markov 
chain; 

{Ws,^,,^i+i{Ys,^,)) ^ X'' ^ (Wi, jl^Yi^ . (122) 

Based on (fT22] i. we apply Lemma [T6] to ^e*''^*'!^'"''^^) 
for I = 1, 2, • • • , i — 1. Then, for / = 1, 2, • • • , L — 1, we have 
the following chains of inequalities : 



,|/i(3',|X^',H'^) 



1 

27re' 



- — e 
27re 

1 



X^^,VKs,_^i,VF, 



1 



27re 27re 



X'\H'i 



27re 






(123) 



Using ( 1123b iteratively for Z = l,2,---,i — 1, we have 



1 2.h('ifi\x'^ ,w^) ^ y^ 

9-77-A — ^ 



27re 



?fci l2~ 



(") 



1=1 



a 



(124) 



Wi 



Combining dllOb . (1121b . and (1124b . we have 



^2/i(" 



27re 



> [QSxi-*Q]..+E 



;=i 



1 ~ e-'-i 



'N: 



Q(s^i.+'Ai];; ,^,,A]'Q 



JVA^(r-X';') 



fcfc 



completing the proof. 
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